[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:13 dist-test-slave-c12x krb5kdc[16304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:13 dist-test-slave-c12x krb5kdc[16304](info): set up 2 sockets
Aug 08 20:56:13 dist-test-slave-c12x krb5kdc[16304](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 20:56:14.928894 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:15.071105 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35219
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:35219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:15.189903 16323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:15.190165 16323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:15.190192 16323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:15.191663 16323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:15.191744 16323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:15.191761 16323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:15.191774 16323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:15.191787 16323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:15.193948 16323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:35219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:15.194368 16323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:15.194599 16323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:15.206488 16331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:15.206491 16337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:15.206797 16330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:15.207530 16323 server_base.cc:1034] running on GCE node
I20260808 20:56:15.208001 16323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:15.208480 16323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:15.209657 16323 hybrid_clock.cc:648] HybridClock initialized: now 1786222575209626 us; error 101 us; skew 500 ppm
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:15.212834 16323 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:15.213599 16323 webserver.cc:460] Webserver started at http://127.15.232.190:40797/ using document root <none> and password file <none>
I20260808 20:56:15.213949 16323 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:15.214030 16323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:15.214222 16323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:15.215773 16323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "8063bda2b36c436886a2fc6dc3fa3217"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "0e9e87e61744345c3e5c8732cc55a52a"
server_key_iv: "8e90d32f08e9525fc4b54f379c3d54af"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.216522 16323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "8063bda2b36c436886a2fc6dc3fa3217"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "0e9e87e61744345c3e5c8732cc55a52a"
server_key_iv: "8e90d32f08e9525fc4b54f379c3d54af"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.218766 16323 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 20:56:15.220089 16351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.220422 16323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:15.220541 16323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "8063bda2b36c436886a2fc6dc3fa3217"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "0e9e87e61744345c3e5c8732cc55a52a"
server_key_iv: "8e90d32f08e9525fc4b54f379c3d54af"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.220639 16323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:15.241500 16323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:15.244645 16323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:15.244786 16323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:15.251602 16323 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:35219
I20260808 20:56:15.251708 16432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:35219 every 8 connection(s)
I20260808 20:56:15.252297 16323 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:15.254194 16433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:15.258385 16433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217: Bootstrap starting.
I20260808 20:56:15.259270 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16323
I20260808 20:56:15.259428 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:15.259784 16433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:15.260281 16433 log.cc:824] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:15.260504 16290 external_mini_cluster.cc:1428] Setting key 24b4adcc3d6e1e761476ad18e67f8f00
I20260808 20:56:15.261492 16433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217: No bootstrap required, opened a new log
I20260808 20:56:15.264215 16433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8063bda2b36c436886a2fc6dc3fa3217" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35219 } }
I20260808 20:56:15.264528 16433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:15.264643 16433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8063bda2b36c436886a2fc6dc3fa3217, State: Initialized, Role: FOLLOWER
I20260808 20:56:15.264856 16433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8063bda2b36c436886a2fc6dc3fa3217" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35219 } }
I20260808 20:56:15.265059 16433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:15.265116 16433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:15.265177 16433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:15.266930 16433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8063bda2b36c436886a2fc6dc3fa3217" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35219 } }
I20260808 20:56:15.267175 16433 leader_election.cc:302] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8063bda2b36c436886a2fc6dc3fa3217; no voters:
I20260808 20:56:15.267478 16433 leader_election.cc:288] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:15.267565 16443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:15.267728 16443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [term 1 LEADER]: Becoming Leader. State: Replica: 8063bda2b36c436886a2fc6dc3fa3217, State: Running, Role: LEADER
I20260808 20:56:15.267845 16443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8063bda2b36c436886a2fc6dc3fa3217" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35219 } }
I20260808 20:56:15.268018 16433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:15.268496 16445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8063bda2b36c436886a2fc6dc3fa3217" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8063bda2b36c436886a2fc6dc3fa3217" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35219 } } }
I20260808 20:56:15.268610 16445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:15.268764 16445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8063bda2b36c436886a2fc6dc3fa3217. Latest consensus state: current_term: 1 leader_uuid: "8063bda2b36c436886a2fc6dc3fa3217" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8063bda2b36c436886a2fc6dc3fa3217" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35219 } } }
I20260808 20:56:15.268843 16445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:15.269042 16448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:15.270862 16448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:15.273937 16448 catalog_manager.cc:1347] Generated new cluster ID: 78830931ffe74b3181e4e8014bac1310
I20260808 20:56:15.274034 16448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:15.284456 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:15.264153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36086 (local address 127.15.232.190:35219)
0808 20:56:15.264418 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:15.264425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:15.264457 (+ 32us) server_negotiation.cc:407] Connection header received
0808 20:56:15.265454 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:15.265462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:15.265566 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:15.265697 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:15.268654 (+ 2957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.271100 (+ 2446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.273257 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.273688 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.273859 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:15.279461 (+ 5602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:15.279468 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:15.279469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:15.279484 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:15.281775 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.282378 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.282382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.282382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.282518 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.282857 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.282858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.282859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.283367 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:15.283659 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:15.283943 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 20:56:15.284169 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260808 20:56:15.286935 16448 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:15.287920 16448 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:15.297024 16448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8063bda2b36c436886a2fc6dc3fa3217: Generated new TSK 0
I20260808 20:56:15.297330 16448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:15.330950 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35219
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:15.428892 16470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:15.429177 16470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:15.429225 16470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:15.430526 16470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:15.430609 16470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:15.430666 16470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:15.432410 16470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35219
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:15.432790 16470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:15.433054 16470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:15.438887 16480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:15.439051 16482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:15.438932 16489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:15.439522 16470 server_base.cc:1034] running on GCE node
I20260808 20:56:15.439760 16470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:15.440052 16470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:15.441233 16470 hybrid_clock.cc:648] HybridClock initialized: now 1786222575441212 us; error 37 us; skew 500 ppm
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:15.444140 16470 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:15.444787 16470 webserver.cc:460] Webserver started at http://127.15.232.129:46083/ using document root <none> and password file <none>
I20260808 20:56:15.445042 16470 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:15.445089 16470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:15.445230 16470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:15.447041 16470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "15944d4950de4698a9d566004c976564"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "8d1c9a388ad143898cb76ad98b0c400c"
server_key_iv: "228438d4f0868681b991510bbc38bac9"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.447666 16470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "15944d4950de4698a9d566004c976564"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "8d1c9a388ad143898cb76ad98b0c400c"
server_key_iv: "228438d4f0868681b991510bbc38bac9"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.449579 16470 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 20:56:15.451261 16498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.451488 16470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:15.451612 16470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "15944d4950de4698a9d566004c976564"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "8d1c9a388ad143898cb76ad98b0c400c"
server_key_iv: "228438d4f0868681b991510bbc38bac9"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.451704 16470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:15.469086 16470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:15.471851 16470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:15.472009 16470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:15.472445 16470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:15.472890 16470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:15.472958 16470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.472985 16470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:15.473027 16470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.485086 16470 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:33101
I20260808 20:56:15.485184 16681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:33101 every 8 connection(s)
I20260808 20:56:15.485560 16470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:15.488911 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16470
I20260808 20:56:15.488999 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:15.489195 16290 external_mini_cluster.cc:1428] Setting key a736b012a0fb69a3a69d40f3a1266a26
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:15.497942 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:15.486597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35219 (local address 127.15.232.129:58269)
0808 20:56:15.486826 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:15.486866 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:15.487477 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:15.487778 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:15.487782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:15.488525 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:15.488769 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:15.488775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.489721 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.489722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:15.490631 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:15.490634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.491056 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.491058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:15.491193 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:15.492012 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:15.492022 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:15.494172 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:15.496324 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:15.496332 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:15.496334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:15.496701 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:15.496932 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:15.496933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:15.496933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:15.497040 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:15.497371 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:15.497376 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:15.497527 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 20:56:15.497615 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 20:56:15.497749 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":125,"threads_started":1}
I20260808 20:56:15.498010 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:15.486622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:58269 (local address 127.15.232.190:35219)
0808 20:56:15.486706 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:15.486708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:15.486965 (+ 257us) server_negotiation.cc:407] Connection header received
0808 20:56:15.487624 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:15.487626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:15.487646 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:15.487698 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:15.488881 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.489624 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.490740 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.490969 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.491047 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:15.494309 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:15.494310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:15.494310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:15.494322 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:15.496226 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.496786 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.496787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.496787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.496860 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.497120 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.497121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.497121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.497294 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:15.497379 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:15.497671 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 20:56:15.497861 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 20:56:15.498785 16686 heartbeater.cc:347] Connected to a master server at 127.15.232.190:35219
I20260808 20:56:15.498893 16686 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:15.499095 16686 heartbeater.cc:511] Master 127.15.232.190:35219 requested a full tablet report, sending...
I20260808 20:56:15.500218 16384 ts_manager.cc:194] Registered new tserver with Master: 15944d4950de4698a9d566004c976564 (127.15.232.129:33101)
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
I20260808 20:56:15.501080 16384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:58269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:15.522846 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35219
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:15.613184 16691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:15.613449 16691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:15.613490 16691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:15.614790 16691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:15.614876 16691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:15.614926 16691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:15.617475 16691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35219
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:15.617992 16691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:15.618522 16691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:15.626372 16700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:15.626401 16704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:15.626436 16701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:15.627055 16691 server_base.cc:1034] running on GCE node
I20260808 20:56:15.627322 16691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:15.628765 16691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:15.632171 16691 hybrid_clock.cc:648] HybridClock initialized: now 1786222575632117 us; error 63 us; skew 500 ppm
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:15.636443 16691 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:15.637135 16691 webserver.cc:460] Webserver started at http://127.15.232.130:36743/ using document root <none> and password file <none>
I20260808 20:56:15.637496 16691 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:15.637580 16691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:15.637733 16691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:15.638978 16691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "a8bad26e0bb54854a0216a9142a24bed"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "e09850fd86f0a4ddb23dff77842fe2c0"
server_key_iv: "d973d4e41f4eaf637b3711c43d23799d"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.639518 16691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "a8bad26e0bb54854a0216a9142a24bed"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "e09850fd86f0a4ddb23dff77842fe2c0"
server_key_iv: "d973d4e41f4eaf637b3711c43d23799d"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.642767 16691 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260808 20:56:15.644223 16715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.644475 16691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:15.644593 16691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "a8bad26e0bb54854a0216a9142a24bed"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "e09850fd86f0a4ddb23dff77842fe2c0"
server_key_iv: "d973d4e41f4eaf637b3711c43d23799d"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.644688 16691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:15.659621 16691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:15.661934 16691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:15.662098 16691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:15.662374 16691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:15.662734 16691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:15.662792 16691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.662835 16691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:15.662863 16691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.673280 16691 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:33131
I20260808 20:56:15.673322 16887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:33131 every 8 connection(s)
I20260808 20:56:15.673708 16691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:15.681993 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16691
I20260808 20:56:15.682102 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:15.682298 16290 external_mini_cluster.cc:1428] Setting key cab27ad7acda8ef79817d55dae05c8ea
I20260808 20:56:15.684522 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:15.674677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:38549 (local address 127.15.232.190:35219)
0808 20:56:15.674800 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:15.674802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:15.675005 (+ 203us) server_negotiation.cc:407] Connection header received
0808 20:56:15.675649 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:15.675650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:15.675663 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:15.675723 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:15.676745 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.677266 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.678327 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.678567 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.678649 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:15.681125 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:15.681127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:15.681127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:15.681136 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:15.682873 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.683373 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.683375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.683376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.683501 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.683798 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.683799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.683799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.683937 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:15.684013 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:15.684333 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 20:56:15.684429 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 20:56:15.684770 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:15.674647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35219 (local address 127.15.232.130:38549)
0808 20:56:15.674869 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:15.674915 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:15.675534 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:15.675799 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:15.675803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:15.676412 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:15.676631 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:15.676636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.677366 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.677367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:15.678208 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:15.678211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.678655 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.678656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:15.678790 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:15.679532 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:15.679542 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:15.681009 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:15.682980 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:15.682988 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:15.682991 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:15.683289 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:15.683595 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:15.683595 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:15.683596 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:15.683716 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:15.684017 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:15.684025 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:15.684189 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 20:56:15.684298 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 20:56:15.684489 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":88,"threads_started":1}
I20260808 20:56:15.685281 16888 heartbeater.cc:347] Connected to a master server at 127.15.232.190:35219
I20260808 20:56:15.685432 16888 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:15.685670 16888 heartbeater.cc:511] Master 127.15.232.190:35219 requested a full tablet report, sending...
I20260808 20:56:15.686216 16384 ts_manager.cc:194] Registered new tserver with Master: a8bad26e0bb54854a0216a9142a24bed (127.15.232.130:33131)
I20260808 20:56:15.686744 16384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:38549
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:15.716658 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35219
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:15.808643 16895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:15.808892 16895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:15.808938 16895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:15.810194 16895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:15.810271 16895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:15.810319 16895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:15.812023 16895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35219
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:15.812296 16895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:15.812561 16895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:15.818195 16908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:15.818280 16911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:15.818194 16905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:15.818558 16895 server_base.cc:1034] running on GCE node
I20260808 20:56:15.818780 16895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:15.819058 16895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:15.820233 16895 hybrid_clock.cc:648] HybridClock initialized: now 1786222575820209 us; error 40 us; skew 500 ppm
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:15.823133 16895 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:15.823745 16895 webserver.cc:460] Webserver started at http://127.15.232.131:33969/ using document root <none> and password file <none>
I20260808 20:56:15.824018 16895 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:15.824097 16895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:15.824258 16895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:15.825446 16895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "865bfa4c3ca64c03aa5d6830a4d2717b"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "feb9b6729c2e3325b90a1963347a4311"
server_key_iv: "76f6364312a72c689a58aaddec0f7c5d"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.825930 16895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "865bfa4c3ca64c03aa5d6830a4d2717b"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "feb9b6729c2e3325b90a1963347a4311"
server_key_iv: "76f6364312a72c689a58aaddec0f7c5d"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.827651 16895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 20:56:15.828559 16930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.828863 16895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:15.828976 16895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "865bfa4c3ca64c03aa5d6830a4d2717b"
format_stamp: "Formatted at 2026-08-08 20:56:15 on dist-test-slave-c12x"
server_key: "feb9b6729c2e3325b90a1963347a4311"
server_key_iv: "76f6364312a72c689a58aaddec0f7c5d"
server_key_version: "encryptionkey@0"
I20260808 20:56:15.829070 16895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:15.838595 16895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:15.841053 16895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:15.841226 16895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:15.841653 16895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:15.842096 16895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:15.842157 16895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.842195 16895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:15.842245 16895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:15.851792 16895 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:44057
I20260808 20:56:15.851830 17092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:44057 every 8 connection(s)
I20260808 20:56:15.852270 16895 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:15.853629 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16895
I20260808 20:56:15.853750 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:15.853919 16290 external_mini_cluster.cc:1428] Setting key d4939c58b604190f932033491e50693b
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:15.863262 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:15.853509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:55971 (local address 127.15.232.190:35219)
0808 20:56:15.853579 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:15.853581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:15.853757 (+ 176us) server_negotiation.cc:407] Connection header received
0808 20:56:15.854351 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:15.854353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:15.854366 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:15.854417 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:15.855509 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.856104 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.857565 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.857784 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.857866 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:15.860382 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:15.860384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:15.860384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:15.860392 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:15.861749 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.862222 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.862224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.862224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.862291 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.862564 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.862565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.862565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.862691 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:15.862768 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:15.863066 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 20:56:15.863157 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 20:56:15.863535 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:15.853401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35219 (local address 127.15.232.131:55971)
0808 20:56:15.853610 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:15.853654 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:15.854226 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:15.854493 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:15.854498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:15.855169 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:15.855375 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:15.855380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.856204 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.856205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:15.857438 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:15.857441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.857875 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.857876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:15.858018 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:15.858558 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:15.858577 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:15.860256 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:15.861879 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:15.861887 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:15.861890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:15.862138 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:15.862365 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:15.862366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:15.862366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:15.862485 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:15.862770 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:15.862779 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:15.862928 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 20:56:15.863040 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 20:56:15.863229 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260808 20:56:15.863997 17093 heartbeater.cc:347] Connected to a master server at 127.15.232.190:35219
I20260808 20:56:15.864105 17093 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:15.864262 17093 heartbeater.cc:511] Master 127.15.232.190:35219 requested a full tablet report, sending...
I20260808 20:56:15.864675 16384 ts_manager.cc:194] Registered new tserver with Master: 865bfa4c3ca64c03aa5d6830a4d2717b (127.15.232.131:44057)
I20260808 20:56:15.865017 16384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:55971
I20260808 20:56:15.866173 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:15.895123 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:15.886413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.15.232.129:33101)
0808 20:56:15.886771 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:15.886777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:15.886788 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:15.886824 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:15.886825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:15.886839 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:15.886928 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:15.887713 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.888465 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.889263 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.889622 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.889704 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:15.892140 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:15.892148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:15.892148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:15.892155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:15.893956 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.894283 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.894285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.894286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.894344 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.894589 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.894590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.894590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.894731 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:15.894851 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:15.894913 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 20:56:15.894979 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":272,"threads_started":1}
W20260808 20:56:15.895841 16545 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36192
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 20:56:15 dist-test-slave-c12x krb5kdc[16304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:15.922099 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:15.914251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36198 (local address 127.15.232.129:33101)
0808 20:56:15.914328 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:15.914330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:15.914364 (+ 34us) server_negotiation.cc:407] Connection header received
0808 20:56:15.914467 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:15.914468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:15.914481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:15.914570 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:15.915163 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.915748 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.916496 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:15.916706 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:15.916786 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:15.919379 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:15.919380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:15.919381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:15.919388 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:15.920887 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.921261 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.921262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.921263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.921321 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:15.921551 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:15.921551 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:15.921552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:15.921655 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:15.921760 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:15.921956 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 20:56:15.922030 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 20:56:15.922956 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16470
I20260808 20:56:15.930717 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16691
I20260808 20:56:15.944628 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16895
I20260808 20:56:15.956923 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16323
2026-08-08T20:56:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2395 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:16 dist-test-slave-c12x krb5kdc[17110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:16 dist-test-slave-c12x krb5kdc[17110](info): set up 2 sockets
Aug 08 20:56:16 dist-test-slave-c12x krb5kdc[17110](info): commencing operation
krb5kdc: starting...
W20260808 20:56:17.221609 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:17 dist-test-slave-c12x krb5kdc[17110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:17.362231 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:40217
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:40217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:17.449972 17134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:17.450209 17134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:17.450233 17134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:17.451689 17134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:17.451762 17134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:17.451782 17134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:17.451797 17134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:17.451808 17134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:17.453614 17134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:40217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:40217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:17.453886 17134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:17.454128 17134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:17.459635 17140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:17.459630 17141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:17.459735 17145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:17.460098 17134 server_base.cc:1034] running on GCE node
I20260808 20:56:17.460377 17134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:17.460734 17134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:17.461918 17134 hybrid_clock.cc:648] HybridClock initialized: now 1786222577461869 us; error 48 us; skew 500 ppm
Aug 08 20:56:17 dist-test-slave-c12x krb5kdc[17110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:17.464825 17134 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:17.465348 17134 webserver.cc:460] Webserver started at http://127.15.232.190:46803/ using document root <none> and password file <none>
I20260808 20:56:17.465586 17134 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:17.465654 17134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:17.465807 17134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:17.466902 17134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "597ca131d6d8429ba56bf8aedb0a4a3f"
format_stamp: "Formatted at 2026-08-08 20:56:17 on dist-test-slave-c12x"
server_key: "d5b10641e8a8c2e3881357826bfdec8d"
server_key_iv: "6f6e1e1e67ace8e107f0baa9bbf448c7"
server_key_version: "encryptionkey@0"
I20260808 20:56:17.467305 17134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "597ca131d6d8429ba56bf8aedb0a4a3f"
format_stamp: "Formatted at 2026-08-08 20:56:17 on dist-test-slave-c12x"
server_key: "d5b10641e8a8c2e3881357826bfdec8d"
server_key_iv: "6f6e1e1e67ace8e107f0baa9bbf448c7"
server_key_version: "encryptionkey@0"
I20260808 20:56:17.468856 17134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:17.469810 17155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:17.470039 17134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:17.470161 17134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "597ca131d6d8429ba56bf8aedb0a4a3f"
format_stamp: "Formatted at 2026-08-08 20:56:17 on dist-test-slave-c12x"
server_key: "d5b10641e8a8c2e3881357826bfdec8d"
server_key_iv: "6f6e1e1e67ace8e107f0baa9bbf448c7"
server_key_version: "encryptionkey@0"
I20260808 20:56:17.470240 17134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:17.479843 17134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:17.482654 17134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:17.483040 17134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:17.487743 17134 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:40217
I20260808 20:56:17.487896 17231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:40217 every 8 connection(s)
I20260808 20:56:17.488204 17134 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:17.489471 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 17134
I20260808 20:56:17.489571 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:17.489713 17232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:17.489746 16290 external_mini_cluster.cc:1428] Setting key ff9b2c6bc282e8c9a2397da841d7c6a7
I20260808 20:56:17.494956 17232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f: Bootstrap starting.
I20260808 20:56:17.495750 17232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:17.496062 17232 log.cc:824] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:17.497241 17232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f: No bootstrap required, opened a new log
Aug 08 20:56:17 dist-test-slave-c12x krb5kdc[17110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:17.502121 17232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597ca131d6d8429ba56bf8aedb0a4a3f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40217 } }
I20260808 20:56:17.502470 17232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:17.502561 17232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597ca131d6d8429ba56bf8aedb0a4a3f, State: Initialized, Role: FOLLOWER
I20260808 20:56:17.502787 17232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597ca131d6d8429ba56bf8aedb0a4a3f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40217 } }
I20260808 20:56:17.502907 17232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:17.502947 17232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:17.503028 17232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:17.504079 17232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597ca131d6d8429ba56bf8aedb0a4a3f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40217 } }
I20260808 20:56:17.504305 17232 leader_election.cc:302] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 597ca131d6d8429ba56bf8aedb0a4a3f; no voters:
I20260808 20:56:17.504384 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:17.490339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.15.232.190:40217)
0808 20:56:17.490524 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:17.490534 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:17.490573 (+ 39us) server_negotiation.cc:407] Connection header received
0808 20:56:17.491569 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:17.491580 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:17.491650 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:17.491780 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:17.492816 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.493760 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.494861 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.495756 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.495997 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:17.499687 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:17.499693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:17.499694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:17.499710 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:17.502343 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:17.502835 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:17.502839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:17.502840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:17.502961 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:17.503201 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:17.503202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:17.503203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:17.503663 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:17.503829 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:17.503967 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 20:56:17.504132 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260808 20:56:17.504510 17232 leader_election.cc:288] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:17.504572 17237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:17.504714 17237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [term 1 LEADER]: Becoming Leader. State: Replica: 597ca131d6d8429ba56bf8aedb0a4a3f, State: Running, Role: LEADER
I20260808 20:56:17.504793 17237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597ca131d6d8429ba56bf8aedb0a4a3f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40217 } }
I20260808 20:56:17.504799 17232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:17.505210 17237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 597ca131d6d8429ba56bf8aedb0a4a3f. Latest consensus state: current_term: 1 leader_uuid: "597ca131d6d8429ba56bf8aedb0a4a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597ca131d6d8429ba56bf8aedb0a4a3f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40217 } } }
I20260808 20:56:17.505275 17237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:17.505746 17245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:17.506328 17245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:17.506915 17240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "597ca131d6d8429ba56bf8aedb0a4a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597ca131d6d8429ba56bf8aedb0a4a3f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40217 } } }
I20260808 20:56:17.510982 17240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:17.509675 17245 catalog_manager.cc:1347] Generated new cluster ID: f1dc6b8920bc4d659540e9d87bf298b8
I20260808 20:56:17.511199 17245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:17.524230 17245 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:17.524864 17245 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:17.530066 17245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 597ca131d6d8429ba56bf8aedb0a4a3f: Generated new TSK 0
I20260808 20:56:17.530278 17245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:17.575130 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40217
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:17.677865 17267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:17.678081 17267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:17.678105 17267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:17.679293 17267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:17.679337 17267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:17.679364 17267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:17.680943 17267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40217
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:17.681200 17267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:17.681447 17267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:17.686828 17277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:17.686836 17275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:17.686827 17273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:17.687191 17267 server_base.cc:1034] running on GCE node
I20260808 20:56:17.687476 17267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:17.687741 17267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:17.688911 17267 hybrid_clock.cc:648] HybridClock initialized: now 1786222577688889 us; error 48 us; skew 500 ppm
Aug 08 20:56:17 dist-test-slave-c12x krb5kdc[17110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:17.691808 17267 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:17.692263 17267 webserver.cc:460] Webserver started at http://127.15.232.129:39585/ using document root <none> and password file <none>
I20260808 20:56:17.692539 17267 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:17.692593 17267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:17.692696 17267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:17.693909 17267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "0d9b71c2da514fd38e77ac505dc9e576"
format_stamp: "Formatted at 2026-08-08 20:56:17 on dist-test-slave-c12x"
server_key: "4bd26d8093398cad1813ea4ffb8b166b"
server_key_iv: "98510d60eb8145e1af17e60a188984d4"
server_key_version: "encryptionkey@0"
I20260808 20:56:17.694376 17267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "0d9b71c2da514fd38e77ac505dc9e576"
format_stamp: "Formatted at 2026-08-08 20:56:17 on dist-test-slave-c12x"
server_key: "4bd26d8093398cad1813ea4ffb8b166b"
server_key_iv: "98510d60eb8145e1af17e60a188984d4"
server_key_version: "encryptionkey@0"
I20260808 20:56:17.695777 17267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 20:56:17.696549 17286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:17.696761 17267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:17.696846 17267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "0d9b71c2da514fd38e77ac505dc9e576"
format_stamp: "Formatted at 2026-08-08 20:56:17 on dist-test-slave-c12x"
server_key: "4bd26d8093398cad1813ea4ffb8b166b"
server_key_iv: "98510d60eb8145e1af17e60a188984d4"
server_key_version: "encryptionkey@0"
I20260808 20:56:17.696902 17267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:17.703401 17267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:17.705471 17267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:17.705657 17267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:17.705890 17267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:17.706218 17267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:17.706279 17267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:17.706317 17267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:17.706343 17267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:17.712802 17267 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:35147
I20260808 20:56:17.712880 17446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:35147 every 8 connection(s)
I20260808 20:56:17.713191 17267 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
Aug 08 20:56:17 dist-test-slave-c12x krb5kdc[17110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:17.722831 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:17.713977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:35925 (local address 127.15.232.190:40217)
0808 20:56:17.714040 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:17.714046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:17.714266 (+ 220us) server_negotiation.cc:407] Connection header received
0808 20:56:17.715368 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:17.715370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:17.715380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:17.715529 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:17.716154 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.716688 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.717676 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.717899 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.717966 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:17.720221 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:17.720223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:17.720223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:17.720229 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:17.721473 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:17.721834 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:17.721835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:17.721835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:17.721890 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:17.722129 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:17.722130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:17.722130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:17.722250 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:17.722365 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:17.722673 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 20:56:17.722761 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 20:56:17.722975 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:17.713951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:40217 (local address 127.15.232.129:35925)
0808 20:56:17.714140 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:17.714183 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:17.715264 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:17.715596 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:17.715599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:17.715863 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:17.716057 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:17.716061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.716777 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.716778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:17.717591 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:17.717593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.717984 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.717985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:17.718099 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:17.718652 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:17.718661 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:17.720137 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:17.721543 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:17.721550 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:17.721551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:17.721765 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:17.721949 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:17.721949 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:17.721950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:17.722053 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:17.722352 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:17.722356 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:17.722490 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 20:56:17.722648 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 20:56:17.722778 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260808 20:56:17.723039 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 17267
I20260808 20:56:17.723119 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:17.723318 16290 external_mini_cluster.cc:1428] Setting key 61f847aab913a6873239c065d1a13c41
I20260808 20:56:17.723589 17447 heartbeater.cc:347] Connected to a master server at 127.15.232.190:40217
I20260808 20:56:17.723703 17447 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:17.723892 17447 heartbeater.cc:511] Master 127.15.232.190:40217 requested a full tablet report, sending...
I20260808 20:56:17.724476 17175 ts_manager.cc:194] Registered new tserver with Master: 0d9b71c2da514fd38e77ac505dc9e576 (127.15.232.129:35147)
I20260808 20:56:17.725380 17175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:35925
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:17.750487 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40217
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:17.838517 17452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:17.838734 17452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:17.838759 17452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:17.839982 17452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:17.840021 17452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:17.840049 17452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:17.841609 17452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40217
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:17.841876 17452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:17.842113 17452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:17.847488 17461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:17.847524 17458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:17.847514 17459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:17.847723 17452 server_base.cc:1034] running on GCE node
I20260808 20:56:17.847972 17452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:17.848244 17452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:17.849448 17452 hybrid_clock.cc:648] HybridClock initialized: now 1786222577849403 us; error 39 us; skew 500 ppm
Aug 08 20:56:17 dist-test-slave-c12x krb5kdc[17110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:17.852149 17452 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:17.852636 17452 webserver.cc:460] Webserver started at http://127.15.232.130:33123/ using document root <none> and password file <none>
I20260808 20:56:17.852886 17452 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:17.852954 17452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:17.853101 17452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:17.854257 17452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "13967f2b2750450886ce78282c11331b"
format_stamp: "Formatted at 2026-08-08 20:56:17 on dist-test-slave-c12x"
server_key: "78f5d083d673b9209cdf1480da15a640"
server_key_iv: "ba40f6806e625334994583b7895ac786"
server_key_version: "encryptionkey@0"
I20260808 20:56:17.854637 17452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "13967f2b2750450886ce78282c11331b"
format_stamp: "Formatted at 2026-08-08 20:56:17 on dist-test-slave-c12x"
server_key: "78f5d083d673b9209cdf1480da15a640"
server_key_iv: "ba40f6806e625334994583b7895ac786"
server_key_version: "encryptionkey@0"
I20260808 20:56:17.856076 17452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:17.856868 17479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:17.857048 17452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:17.857167 17452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "13967f2b2750450886ce78282c11331b"
format_stamp: "Formatted at 2026-08-08 20:56:17 on dist-test-slave-c12x"
server_key: "78f5d083d673b9209cdf1480da15a640"
server_key_iv: "ba40f6806e625334994583b7895ac786"
server_key_version: "encryptionkey@0"
I20260808 20:56:17.857244 17452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:17.879233 17452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:17.882059 17452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:17.882266 17452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:17.882565 17452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:17.883046 17452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:17.883105 17452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:17.883144 17452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:17.883175 17452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:17.890389 17452 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:41865
I20260808 20:56:17.890522 17639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:41865 every 8 connection(s)
I20260808 20:56:17.891037 17452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:17.896981 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 17452
I20260808 20:56:17.897060 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:17.897240 16290 external_mini_cluster.cc:1428] Setting key 52dffaa9fc59930ab6f53eaaf03f8c6a
Aug 08 20:56:17 dist-test-slave-c12x krb5kdc[17110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:17.901609 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:17.891976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:39687 (local address 127.15.232.190:40217)
0808 20:56:17.892053 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:17.892056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:17.892280 (+ 224us) server_negotiation.cc:407] Connection header received
0808 20:56:17.893343 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:17.893344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:17.893355 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:17.893421 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:17.894076 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.894580 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.895625 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.895815 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.895878 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:17.898335 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:17.898336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:17.898336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:17.898344 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:17.900116 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:17.900616 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:17.900617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:17.900617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:17.900688 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:17.900958 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:17.900959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:17.900960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:17.901093 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:17.901149 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:17.901415 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 20:56:17.901507 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 20:56:17.901801 17643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:17.891915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:40217 (local address 127.15.232.130:39687)
0808 20:56:17.892156 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:17.892196 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:17.893211 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:17.893502 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:17.893505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:17.893776 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:17.893992 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:17.893996 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.894663 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.894664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:17.895549 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:17.895552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:17.895895 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:17.895896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:17.896015 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:17.896508 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:17.896516 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:17.898243 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:17.900197 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:17.900203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:17.900205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:17.900520 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:17.900783 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:17.900784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:17.900784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:17.900893 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:17.901149 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:17.901153 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:17.901276 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 20:56:17.901387 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 20:56:17.901587 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":110,"threads_started":1}
I20260808 20:56:17.902354 17641 heartbeater.cc:347] Connected to a master server at 127.15.232.190:40217
I20260808 20:56:17.902494 17641 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:17.902714 17641 heartbeater.cc:511] Master 127.15.232.190:40217 requested a full tablet report, sending...
I20260808 20:56:17.903152 17175 ts_manager.cc:194] Registered new tserver with Master: 13967f2b2750450886ce78282c11331b (127.15.232.130:41865)
I20260808 20:56:17.903782 17175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:39687
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:17.926395 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40217
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:18.013532 17646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:18.013787 17646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:18.013829 17646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:18.015054 17646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:18.015121 17646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:18.015169 17646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:18.016827 17646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40217
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:18.017117 17646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:18.017357 17646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:18.023346 17654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:18.023411 17646 server_base.cc:1034] running on GCE node
W20260808 20:56:18.023568 17653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:18.023548 17657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:18.023910 17646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:18.024184 17646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:18.025359 17646 hybrid_clock.cc:648] HybridClock initialized: now 1786222578025331 us; error 49 us; skew 500 ppm
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222578, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:18.028190 17646 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:18.028656 17646 webserver.cc:460] Webserver started at http://127.15.232.131:44759/ using document root <none> and password file <none>
I20260808 20:56:18.028900 17646 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:18.028966 17646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:18.029132 17646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:18.030230 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "ff34381b874e46a9b191f649c692fb86"
format_stamp: "Formatted at 2026-08-08 20:56:18 on dist-test-slave-c12x"
server_key: "949196e3da11f4dd772f3198eacb4f26"
server_key_iv: "341263b46891fadd831f576448dc29a5"
server_key_version: "encryptionkey@0"
I20260808 20:56:18.030637 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "ff34381b874e46a9b191f649c692fb86"
format_stamp: "Formatted at 2026-08-08 20:56:18 on dist-test-slave-c12x"
server_key: "949196e3da11f4dd772f3198eacb4f26"
server_key_iv: "341263b46891fadd831f576448dc29a5"
server_key_version: "encryptionkey@0"
I20260808 20:56:18.032367 17646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:18.033210 17666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:18.033452 17646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:18.033561 17646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "ff34381b874e46a9b191f649c692fb86"
format_stamp: "Formatted at 2026-08-08 20:56:18 on dist-test-slave-c12x"
server_key: "949196e3da11f4dd772f3198eacb4f26"
server_key_iv: "341263b46891fadd831f576448dc29a5"
server_key_version: "encryptionkey@0"
I20260808 20:56:18.033636 17646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:18.050889 17646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:18.053224 17646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:18.053403 17646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:18.053670 17646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:18.053992 17646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:18.054049 17646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:18.054091 17646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:18.054134 17646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:18.060688 17646 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:38623
I20260808 20:56:18.060742 17832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:38623 every 8 connection(s)
I20260808 20:56:18.061115 17646 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:18.062990 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 17646
I20260808 20:56:18.063107 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:18.063277 16290 external_mini_cluster.cc:1428] Setting key bebbbcc9f03bdef75d051bb2c0e1650c
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222578, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:18.071388 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.061998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:36033 (local address 127.15.232.190:40217)
0808 20:56:18.062086 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.062088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.062199 (+ 111us) server_negotiation.cc:407] Connection header received
0808 20:56:18.063209 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.063210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.063225 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.063301 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:18.064194 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.064701 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.065721 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.065921 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.065997 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.068268 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:18.068270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:18.068270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:18.068278 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:18.069708 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.070234 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.070236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.070236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.070303 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.070656 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.070657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.070658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.070790 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:18.070873 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.071173 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.071275 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 20:56:18.071615 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.061913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:40217 (local address 127.15.232.131:36033)
0808 20:56:18.062086 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:18.062124 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:18.063080 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:18.063398 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:18.063401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:18.063817 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:18.064107 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.064114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.064799 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.064800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.065629 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.065632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.066004 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.066006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.066161 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.066710 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:18.066719 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:18.068173 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:18.069833 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:18.069842 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:18.069844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:18.070132 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:18.070445 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:18.070446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:18.070447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:18.070570 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:18.070868 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:18.070876 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:18.071031 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 20:56:18.071149 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 20:56:18.071323 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260808 20:56:18.072157 17834 heartbeater.cc:347] Connected to a master server at 127.15.232.190:40217
I20260808 20:56:18.072278 17834 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:18.072491 17834 heartbeater.cc:511] Master 127.15.232.190:40217 requested a full tablet report, sending...
I20260808 20:56:18.072875 17175 ts_manager.cc:194] Registered new tserver with Master: ff34381b874e46a9b191f649c692fb86 (127.15.232.131:38623)
I20260808 20:56:18.073478 17175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:36033
I20260808 20:56:18.075037 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:56:18.083045 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.077154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.15.232.190:40217)
0808 20:56:18.077216 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.077218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.077320 (+ 102us) server_negotiation.cc:407] Connection header received
0808 20:56:18.077362 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.077362 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.077371 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.077415 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:18.077948 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.078412 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.079139 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.079331 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.079400 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.080741 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:18.080742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:18.080742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:18.080748 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:18.081988 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.082354 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.082354 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.082355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.082405 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.082575 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.082576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.082576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.082723 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:18.082798 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.082882 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.082953 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 20:56:18.086019 17175 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:18.088023 17175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:18.095825 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.091798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.15.232.130:41865)
0808 20:56:18.091968 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.091973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.091985 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:56:18.092029 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.092031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.092047 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.092122 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:18.092235 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.093152 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.094606 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.095354 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.095466 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.095533 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.095691 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.095737 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260808 20:56:18.095849 17853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.091324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:41865 (local address 127.0.0.1:48834)
0808 20:56:18.091563 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:18.091604 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:18.091689 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:18.092126 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:18.092127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:18.092128 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:18.092179 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.092181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.093249 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.093250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.094508 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.094511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.095483 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.095485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.095568 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.095586 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 20:56:18.095660 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 20:56:18.095729 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":122,"threads_started":1}
I20260808 20:56:18.096024 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.091566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37470 (local address 127.15.232.129:35147)
0808 20:56:18.091846 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.091851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.091860 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:18.091892 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.091893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.091911 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.091997 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:18.092185 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.093488 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.094716 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.095477 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.095540 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.095590 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.095892 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.095947 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":110,"threads_started":1}
I20260808 20:56:18.096177 17851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.091333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35147 (local address 127.0.0.1:37470)
0808 20:56:18.091648 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:18.091657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:18.091704 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:18.092004 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:18.092008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:18.092011 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:18.092094 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.092098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.093599 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.093600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.094629 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.094631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.095625 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.095626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.095691 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.095695 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:18.095744 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 20:56:18.095786 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":93,"threads_started":1}
I20260808 20:56:18.096697 17852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.091339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:38623 (local address 127.0.0.1:54184)
0808 20:56:18.091983 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:18.091995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:18.092049 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:18.092293 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:18.092294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:18.092295 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:18.092331 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.092332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.094534 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.094535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.095532 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.095534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.096471 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.096472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.096541 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.096547 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 20:56:18.096614 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 20:56:18.096652 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":88,"threads_started":1}
I20260808 20:56:18.096740 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.091657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.15.232.131:38623)
0808 20:56:18.092143 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.092148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.092159 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:18.092196 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.092198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.092214 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.092296 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:18.092384 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.093484 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.095621 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.096385 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.096470 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.096530 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.096628 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.096668 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":90,"threads_started":1}
I20260808 20:56:18.097035 17355 tablet_service.cc:1467] Processing CreateTablet for tablet 61abcb7d6b80403da990b4b4526878a9 (DEFAULT_TABLE table=test-table [id=3a1eaffed0374a4da64ff49c306b9302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:18.097450 17355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61abcb7d6b80403da990b4b4526878a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:18.097362 17737 tablet_service.cc:1467] Processing CreateTablet for tablet 61abcb7d6b80403da990b4b4526878a9 (DEFAULT_TABLE table=test-table [id=3a1eaffed0374a4da64ff49c306b9302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:18.097689 17737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61abcb7d6b80403da990b4b4526878a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:18.099960 17548 tablet_service.cc:1467] Processing CreateTablet for tablet 61abcb7d6b80403da990b4b4526878a9 (DEFAULT_TABLE table=test-table [id=3a1eaffed0374a4da64ff49c306b9302]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:18.100373 17548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61abcb7d6b80403da990b4b4526878a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:18.101442 17857 tablet_bootstrap.cc:492] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576: Bootstrap starting.
I20260808 20:56:18.102291 17857 tablet_bootstrap.cc:654] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:18.102975 17858 tablet_bootstrap.cc:492] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86: Bootstrap starting.
I20260808 20:56:18.103108 17857 log.cc:824] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:18.103952 17858 tablet_bootstrap.cc:654] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:18.103912 17860 tablet_bootstrap.cc:492] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b: Bootstrap starting.
I20260808 20:56:18.104693 17860 tablet_bootstrap.cc:654] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:18.104679 17858 log.cc:824] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:18.105419 17860 log.cc:824] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:18.105691 17857 tablet_bootstrap.cc:492] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576: No bootstrap required, opened a new log
I20260808 20:56:18.105711 17858 tablet_bootstrap.cc:492] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86: No bootstrap required, opened a new log
I20260808 20:56:18.105789 17858 ts_tablet_manager.cc:1397] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 20:56:18.105789 17857 ts_tablet_manager.cc:1397] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 20:56:18.106328 17860 tablet_bootstrap.cc:492] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b: No bootstrap required, opened a new log
I20260808 20:56:18.106393 17860 ts_tablet_manager.cc:1397] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 20:56:18.107807 17857 raft_consensus.cc:348] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } }
I20260808 20:56:18.108021 17857 raft_consensus.cc:374] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:18.108068 17857 raft_consensus.cc:729] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9b71c2da514fd38e77ac505dc9e576, State: Initialized, Role: FOLLOWER
I20260808 20:56:18.108181 17857 consensus_queue.cc:260] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } }
I20260808 20:56:18.108234 17858 raft_consensus.cc:348] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } }
I20260808 20:56:18.108425 17858 raft_consensus.cc:374] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:18.108467 17858 raft_consensus.cc:729] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff34381b874e46a9b191f649c692fb86, State: Initialized, Role: FOLLOWER
I20260808 20:56:18.108554 17857 ts_tablet_manager.cc:1428] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 20:56:18.108646 17858 consensus_queue.cc:260] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } }
I20260808 20:56:18.108722 17447 heartbeater.cc:503] Master 127.15.232.190:40217 was elected leader, sending a full tablet report...
I20260808 20:56:18.108697 17860 raft_consensus.cc:348] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } }
I20260808 20:56:18.108878 17860 raft_consensus.cc:374] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:18.108920 17860 raft_consensus.cc:729] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13967f2b2750450886ce78282c11331b, State: Initialized, Role: FOLLOWER
I20260808 20:56:18.109078 17858 ts_tablet_manager.cc:1428] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260808 20:56:18.109067 17860 consensus_queue.cc:260] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } }
I20260808 20:56:18.109141 17834 heartbeater.cc:503] Master 127.15.232.190:40217 was elected leader, sending a full tablet report...
I20260808 20:56:18.109385 17641 heartbeater.cc:503] Master 127.15.232.190:40217 was elected leader, sending a full tablet report...
I20260808 20:56:18.109714 17860 ts_tablet_manager.cc:1428] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 20:56:18.133602 17864 raft_consensus.cc:482] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:18.133755 17864 raft_consensus.cc:504] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } }
I20260808 20:56:18.134317 17864 leader_election.cc:288] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13967f2b2750450886ce78282c11331b (127.15.232.130:41865), ff34381b874e46a9b191f649c692fb86 (127.15.232.131:38623)
I20260808 20:56:18.138167 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.134752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:40885 (local address 127.15.232.130:41865)
0808 20:56:18.134896 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.134898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.134941 (+ 43us) server_negotiation.cc:407] Connection header received
0808 20:56:18.135012 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.135013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.135023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.135103 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:18.135358 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.135956 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.137107 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.137732 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.137808 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.137817 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.138051 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.138093 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 20:56:18.138170 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.134582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:38623 (local address 127.15.232.129:50723)
0808 20:56:18.134651 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:18.134668 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:18.134737 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:18.135011 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:18.135012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:18.135014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:18.135122 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.135125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.135957 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.135958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.136962 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.136964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.137856 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.137857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.137975 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.137978 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:18.138039 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 20:56:18.138087 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":33}
I20260808 20:56:18.138217 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.134779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:50723 (local address 127.15.232.131:38623)
0808 20:56:18.134867 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.134870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.134882 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:56:18.134918 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.134919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.134930 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.134991 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:18.135226 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.135845 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.137059 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.137728 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.137807 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.137817 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.138111 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.138159 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 20:56:18.138494 17868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.134608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:41865 (local address 127.15.232.129:40885)
0808 20:56:18.134863 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:18.134875 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:18.134941 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:18.135155 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:18.135156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:18.135158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:18.135271 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.135273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.136032 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.136032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.136981 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:18.136984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.137857 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.137857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:18.137947 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.137951 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:18.138321 (+ 370us) client_negotiation.cc:240] Negotiation successful
0808 20:56:18.138373 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":5,"thread_start_us":182,"threads_started":1}
I20260808 20:56:18.138604 17769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61abcb7d6b80403da990b4b4526878a9" candidate_uuid: "0d9b71c2da514fd38e77ac505dc9e576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff34381b874e46a9b191f649c692fb86" is_pre_election: true
I20260808 20:56:18.138875 17769 raft_consensus.cc:2454] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9b71c2da514fd38e77ac505dc9e576 in term 0.
I20260808 20:56:18.138974 17573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61abcb7d6b80403da990b4b4526878a9" candidate_uuid: "0d9b71c2da514fd38e77ac505dc9e576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13967f2b2750450886ce78282c11331b" is_pre_election: true
I20260808 20:56:18.139161 17573 raft_consensus.cc:2454] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9b71c2da514fd38e77ac505dc9e576 in term 0.
I20260808 20:56:18.139230 17292 leader_election.cc:302] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9b71c2da514fd38e77ac505dc9e576, ff34381b874e46a9b191f649c692fb86; no voters:
I20260808 20:56:18.139402 17864 raft_consensus.cc:2781] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:18.139504 17864 raft_consensus.cc:482] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:18.139540 17864 raft_consensus.cc:3037] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:18.140342 17864 raft_consensus.cc:504] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } }
I20260808 20:56:18.140502 17864 leader_election.cc:288] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [CANDIDATE]: Term 1 election: Requested vote from peers 13967f2b2750450886ce78282c11331b (127.15.232.130:41865), ff34381b874e46a9b191f649c692fb86 (127.15.232.131:38623)
I20260808 20:56:18.140702 17573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61abcb7d6b80403da990b4b4526878a9" candidate_uuid: "0d9b71c2da514fd38e77ac505dc9e576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13967f2b2750450886ce78282c11331b"
I20260808 20:56:18.140722 17769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61abcb7d6b80403da990b4b4526878a9" candidate_uuid: "0d9b71c2da514fd38e77ac505dc9e576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff34381b874e46a9b191f649c692fb86"
I20260808 20:56:18.140805 17769 raft_consensus.cc:3037] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:18.140812 17573 raft_consensus.cc:3037] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:18.141618 17769 raft_consensus.cc:2454] T 61abcb7d6b80403da990b4b4526878a9 P ff34381b874e46a9b191f649c692fb86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9b71c2da514fd38e77ac505dc9e576 in term 1.
W20260808 20:56:18.141803 17642 tablet.cc:2369] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:18.141842 17573 raft_consensus.cc:2454] T 61abcb7d6b80403da990b4b4526878a9 P 13967f2b2750450886ce78282c11331b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9b71c2da514fd38e77ac505dc9e576 in term 1.
I20260808 20:56:18.142037 17288 leader_election.cc:302] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9b71c2da514fd38e77ac505dc9e576, 13967f2b2750450886ce78282c11331b; no voters:
I20260808 20:56:18.142129 17864 raft_consensus.cc:2781] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:18.142329 17864 raft_consensus.cc:686] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9b71c2da514fd38e77ac505dc9e576, State: Running, Role: LEADER
I20260808 20:56:18.142501 17864 consensus_queue.cc:237] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } }
I20260808 20:56:18.143559 17174 catalog_manager.cc:5495] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9b71c2da514fd38e77ac505dc9e576 (127.15.232.129). New cstate: current_term: 1 leader_uuid: "0d9b71c2da514fd38e77ac505dc9e576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13967f2b2750450886ce78282c11331b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9b71c2da514fd38e77ac505dc9e576" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff34381b874e46a9b191f649c692fb86" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38623 } health_report { overall_health: UNKNOWN } } }
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:18.159623 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.151256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37476 (local address 127.15.232.129:35147)
0808 20:56:18.151330 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.151332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.151460 (+ 128us) server_negotiation.cc:407] Connection header received
0808 20:56:18.151774 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.151776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.151788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.151862 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:18.152459 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.152979 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.153729 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.153930 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.154011 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.156511 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:18.156524 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:18.156524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:18.156531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:18.158235 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.158608 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.158611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.158612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.158689 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.159011 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.159012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.159012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.159168 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:18.159284 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.159403 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.159540 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222578, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222578, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:18.186617 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.178550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.15.232.129:35147)
0808 20:56:18.178654 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.178656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.178688 (+ 32us) server_negotiation.cc:407] Connection header received
0808 20:56:18.178726 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.178727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.178737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.178786 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:18.179426 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.180017 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.180839 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.181058 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.181117 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.183726 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:18.183728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:18.183728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:18.183737 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:18.185133 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.185481 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.185482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.185482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.185540 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.185807 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.185810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.185810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.186029 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:18.186268 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.186331 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.186418 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:18.213160 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:18.205118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37490 (local address 127.15.232.129:35147)
0808 20:56:18.205191 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:18.205194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:18.205231 (+ 37us) server_negotiation.cc:407] Connection header received
0808 20:56:18.205334 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:18.205335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:18.205348 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:18.205436 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:18.206176 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.206689 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.207529 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:18.207725 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:18.207790 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:18.210246 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:18.210247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:18.210247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:18.210255 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:18.212014 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.212369 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.212370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.212371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.212433 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:18.212656 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:18.212657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:18.212657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:18.212779 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:18.212895 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:18.213024 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 20:56:18.213091 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 20:56:18.213902 17448 tablet.cc:2369] T 61abcb7d6b80403da990b4b4526878a9 P 0d9b71c2da514fd38e77ac505dc9e576: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 20:56:18.214090 17326 tablet_service.cc:2908] Rejecting scan request for tablet 61abcb7d6b80403da990b4b4526878a9: Uninitialized: clean time has not yet been initialized
I20260808 20:56:18.215142 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 17267
I20260808 20:56:18.228480 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 17452
I20260808 20:56:18.239593 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 17646
I20260808 20:56:18.251014 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 17134
2026-08-08T20:56:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2296 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17893](info): set up 2 sockets
Aug 08 20:56:18 dist-test-slave-c12x krb5kdc[17893](info): commencing operation
krb5kdc: starting...
W20260808 20:56:19.525717 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:19 dist-test-slave-c12x krb5kdc[17893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:19.659574 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:39245
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:39245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:19.744509 17910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:19.744733 17910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:19.744760 17910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:19.745985 17910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:19.746029 17910 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 20:56:19.746043 17910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:19.746057 17910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:19.746068 17910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:19.746106 17910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:19.747741 17910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:39245
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:19.748030 17910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:19.748282 17910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:19.753862 17917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:19.753875 17920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:19.753916 17910 server_base.cc:1034] running on GCE node
W20260808 20:56:19.753875 17916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:19.754314 17910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:19.754647 17910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:19.755792 17910 hybrid_clock.cc:648] HybridClock initialized: now 1786222579755777 us; error 35 us; skew 500 ppm
Aug 08 20:56:19 dist-test-slave-c12x krb5kdc[17893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222579, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:19.763746 17910 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:19.764273 17910 webserver.cc:460] Webserver started at http://127.15.232.190:46283/ using document root <none> and password file <none>
I20260808 20:56:19.764518 17910 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:19.764564 17910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:19.764784 17910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:19.765985 17910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "3984ab3e4c6f443eab0496f35f0d9a59"
format_stamp: "Formatted at 2026-08-08 20:56:19 on dist-test-slave-c12x"
server_key: "3c289866985a70024ab8cd759215a324"
server_key_iv: "34881dc1af29d202b65f4ea9a266b6a0"
server_key_version: "encryptionkey@0"
I20260808 20:56:19.766409 17910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "3984ab3e4c6f443eab0496f35f0d9a59"
format_stamp: "Formatted at 2026-08-08 20:56:19 on dist-test-slave-c12x"
server_key: "3c289866985a70024ab8cd759215a324"
server_key_iv: "34881dc1af29d202b65f4ea9a266b6a0"
server_key_version: "encryptionkey@0"
I20260808 20:56:19.767891 17910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:19.768741 17931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:19.769011 17910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:19.769129 17910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "3984ab3e4c6f443eab0496f35f0d9a59"
format_stamp: "Formatted at 2026-08-08 20:56:19 on dist-test-slave-c12x"
server_key: "3c289866985a70024ab8cd759215a324"
server_key_iv: "34881dc1af29d202b65f4ea9a266b6a0"
server_key_version: "encryptionkey@0"
I20260808 20:56:19.769212 17910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:19.797472 17910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:19.799953 17910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:19.800130 17910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:19.805420 17910 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:39245
I20260808 20:56:19.805510 18015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:39245 every 8 connection(s)
I20260808 20:56:19.805831 17910 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:19.806504 18016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:19.809077 18016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59: Bootstrap starting.
I20260808 20:56:19.809720 18016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:19.810029 18016 log.cc:824] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:19.810767 18016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59: No bootstrap required, opened a new log
I20260808 20:56:19.813232 18016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3984ab3e4c6f443eab0496f35f0d9a59" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39245 } }
I20260808 20:56:19.813409 18016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:19.813462 18016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3984ab3e4c6f443eab0496f35f0d9a59, State: Initialized, Role: FOLLOWER
I20260808 20:56:19.813614 18016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3984ab3e4c6f443eab0496f35f0d9a59" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39245 } }
I20260808 20:56:19.813699 18016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:19.813745 18016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:19.813796 18016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:19.814391 18016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3984ab3e4c6f443eab0496f35f0d9a59" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39245 } }
I20260808 20:56:19.814640 18016 leader_election.cc:302] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3984ab3e4c6f443eab0496f35f0d9a59; no voters:
I20260808 20:56:19.814884 18016 leader_election.cc:288] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:19.814924 18020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:19.815157 18020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [term 1 LEADER]: Becoming Leader. State: Replica: 3984ab3e4c6f443eab0496f35f0d9a59, State: Running, Role: LEADER
I20260808 20:56:19.815193 18016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:19.815253 18020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3984ab3e4c6f443eab0496f35f0d9a59" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39245 } }
I20260808 20:56:19.815716 18022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3984ab3e4c6f443eab0496f35f0d9a59. Latest consensus state: current_term: 1 leader_uuid: "3984ab3e4c6f443eab0496f35f0d9a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3984ab3e4c6f443eab0496f35f0d9a59" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39245 } } }
I20260808 20:56:19.815814 18022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:19.815704 18021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3984ab3e4c6f443eab0496f35f0d9a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3984ab3e4c6f443eab0496f35f0d9a59" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39245 } } }
I20260808 20:56:19.815905 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 17910
I20260808 20:56:19.815971 18021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:19.816040 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:19.816072 18029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:19.816222 16290 external_mini_cluster.cc:1428] Setting key 1602b24cb2705a286092e75fb83f890e
I20260808 20:56:19.816659 18029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:19.818408 18029 catalog_manager.cc:1347] Generated new cluster ID: fd361075313b424c925d77cab3175e99
I20260808 20:56:19.818465 18029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 20:56:19 dist-test-slave-c12x krb5kdc[17893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:19.826733 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:19.816810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41378 (local address 127.15.232.190:39245)
0808 20:56:19.817113 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:19.817125 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:19.817164 (+ 39us) server_negotiation.cc:407] Connection header received
0808 20:56:19.817389 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:19.817397 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:19.817458 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:19.818407 (+ 949us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:19.818545 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:19.819516 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:19.820313 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:19.820753 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:19.820880 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:19.823320 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:19.823324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:19.823324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:19.823338 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:19.825118 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:19.825511 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:19.825514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:19.825514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:19.825628 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:19.825837 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:19.825838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:19.825838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:19.826097 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:19.826273 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:19.826354 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 20:56:19.826526 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":75,"threads_started":1}
I20260808 20:56:19.831234 18029 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:19.831882 18029 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:19.841197 18029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59: Generated new TSK 0
I20260808 20:56:19.841421 18029 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:19.874735 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:39245
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:19.977249 18052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:19.977500 18052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:19.977547 18052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:19.978787 18052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:19.978845 18052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:19.978883 18052 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 20:56:19.978921 18052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:19.980532 18052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.190:39245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:19.980813 18052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:19.981062 18052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:19.986290 18065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:19.986285 18063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:19.986316 18062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:19.986691 18052 server_base.cc:1034] running on GCE node
I20260808 20:56:19.986919 18052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:19.987185 18052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:19.988377 18052 hybrid_clock.cc:648] HybridClock initialized: now 1786222579988333 us; error 61 us; skew 500 ppm
Aug 08 20:56:19 dist-test-slave-c12x krb5kdc[17893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222579, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:19.991067 18052 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:19.991583 18052 webserver.cc:460] Webserver started at http://127.15.232.129:43085/ using document root <none> and password file <none>
I20260808 20:56:19.991814 18052 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:19.991880 18052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:19.992051 18052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:19.993216 18052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "2b02749487f843a0bd0541ef7889d677"
format_stamp: "Formatted at 2026-08-08 20:56:19 on dist-test-slave-c12x"
server_key: "7b10f4a8a3dbddc27dbd608bb6c0c71c"
server_key_iv: "3e83fede174cc3ce603dc3be136b33a5"
server_key_version: "encryptionkey@0"
I20260808 20:56:19.993595 18052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "2b02749487f843a0bd0541ef7889d677"
format_stamp: "Formatted at 2026-08-08 20:56:19 on dist-test-slave-c12x"
server_key: "7b10f4a8a3dbddc27dbd608bb6c0c71c"
server_key_iv: "3e83fede174cc3ce603dc3be136b33a5"
server_key_version: "encryptionkey@0"
I20260808 20:56:19.995002 18052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:19.995792 18078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:19.995965 18052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:19.996071 18052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "2b02749487f843a0bd0541ef7889d677"
format_stamp: "Formatted at 2026-08-08 20:56:19 on dist-test-slave-c12x"
server_key: "7b10f4a8a3dbddc27dbd608bb6c0c71c"
server_key_iv: "3e83fede174cc3ce603dc3be136b33a5"
server_key_version: "encryptionkey@0"
I20260808 20:56:19.996155 18052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:20.032418 18052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:20.034674 18052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:20.034894 18052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:20.035566 18052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:20.035629 18052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:20.035670 18052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:20.035723 18052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 20:56:20 dist-test-slave-c12x krb5kdc[17893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222579, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:20.044130 18052 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:38529
I20260808 20:56:20.044528 18052 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:20.044697 18253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:38529 every 8 connection(s)
I20260808 20:56:20.047585 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.036208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:34037 (local address 127.15.232.190:39245)
0808 20:56:20.036291 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.036294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.036525 (+ 231us) server_negotiation.cc:407] Connection header received
0808 20:56:20.037329 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.037330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.037341 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.037400 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:20.038451 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.039267 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.041041 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.041408 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.041514 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.044594 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:20.044596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:20.044596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:20.044610 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:20.046121 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.046541 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.046542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.046542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.046604 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.046863 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.046864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.046864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.046993 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:20.047063 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.047353 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.047488 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 20:56:20.047773 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.036179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:39245 (local address 127.15.232.129:34037)
0808 20:56:20.036370 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.036422 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.037187 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.037504 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.037510 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.037962 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:20.038294 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.038304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.039391 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.039393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.040901 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.040907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.041540 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.041541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.041725 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.042917 (+ 1192us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:20.042926 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:20.044486 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:20.046209 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.046217 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.046219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.046459 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.046671 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.046672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.046672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.046790 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.047066 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:20.047074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:20.047220 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.047329 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.047533 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260808 20:56:20.048287 18254 heartbeater.cc:347] Connected to a master server at 127.15.232.190:39245
I20260808 20:56:20.048383 18254 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:20.048597 18254 heartbeater.cc:511] Master 127.15.232.190:39245 requested a full tablet report, sending...
I20260808 20:56:20.049204 17951 ts_manager.cc:194] Registered new tserver with Master: 2b02749487f843a0bd0541ef7889d677 (127.15.232.129:38529)
I20260808 20:56:20.050213 17951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:34037
I20260808 20:56:20.052155 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 18052
I20260808 20:56:20.052301 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:20.052508 16290 external_mini_cluster.cc:1428] Setting key 513ade8289f1f7e857974aa19ceaed36
I20260808 20:56:20.056439 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.049582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:56113 (local address 127.15.232.190:39245)
0808 20:56:20.049625 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.049627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.049753 (+ 126us) server_negotiation.cc:407] Connection header received
0808 20:56:20.049779 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.049780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.049789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.049848 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:20.050227 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.050910 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.051866 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.052165 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.052280 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.053322 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:20.053323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:20.053323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:20.053342 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:20.055202 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.055731 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.055732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.055732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.055788 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.055999 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.055999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.056000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.056094 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:20.056140 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.056306 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.056374 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:56:20.056576 18264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.049491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:39245 (local address 127.15.232.129:56113)
0808 20:56:20.049629 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.049652 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.049714 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.049859 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.049860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.050057 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:20.050132 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.050134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.051033 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.051034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.051761 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.051765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.052285 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.052286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.052365 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.052882 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:20.052887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:20.053217 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:20.055304 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.055306 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.055306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.055627 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.055856 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.055857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.055858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.055938 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.056150 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:20.056151 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:20.056222 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.056294 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.056429 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:20.083164 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:39245
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:20.168442 18267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:20.168674 18267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:20.168700 18267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:20.169919 18267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:20.169965 18267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:20.170015 18267 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 20:56:20.170053 18267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:20.171576 18267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.190:39245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:20.171835 18267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:20.172068 18267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:20.177537 18273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:20.177623 18267 server_base.cc:1034] running on GCE node
W20260808 20:56:20.177486 18275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:20.177517 18277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:20.177901 18267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:20.178172 18267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:20.179327 18267 hybrid_clock.cc:648] HybridClock initialized: now 1786222580179313 us; error 28 us; skew 500 ppm
Aug 08 20:56:20 dist-test-slave-c12x krb5kdc[17893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222580, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:20.181924 18267 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:20.182404 18267 webserver.cc:460] Webserver started at http://127.15.232.130:42299/ using document root <none> and password file <none>
I20260808 20:56:20.182643 18267 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:20.182711 18267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:20.182866 18267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:20.184044 18267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "1daa4de880d0483aad16e14a395c0e88"
format_stamp: "Formatted at 2026-08-08 20:56:20 on dist-test-slave-c12x"
server_key: "e2886af5d72c9fdf610d8a19042c4523"
server_key_iv: "16e91ad32b0b7139fce8def70183c15a"
server_key_version: "encryptionkey@0"
I20260808 20:56:20.184424 18267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "1daa4de880d0483aad16e14a395c0e88"
format_stamp: "Formatted at 2026-08-08 20:56:20 on dist-test-slave-c12x"
server_key: "e2886af5d72c9fdf610d8a19042c4523"
server_key_iv: "16e91ad32b0b7139fce8def70183c15a"
server_key_version: "encryptionkey@0"
I20260808 20:56:20.186023 18267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:20.186898 18289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:20.187094 18267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:20.187204 18267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "1daa4de880d0483aad16e14a395c0e88"
format_stamp: "Formatted at 2026-08-08 20:56:20 on dist-test-slave-c12x"
server_key: "e2886af5d72c9fdf610d8a19042c4523"
server_key_iv: "16e91ad32b0b7139fce8def70183c15a"
server_key_version: "encryptionkey@0"
I20260808 20:56:20.187281 18267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:20.201371 18267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:20.203485 18267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:20.203657 18267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:20.204316 18267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:20.204368 18267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:20.204406 18267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:20.204424 18267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 20:56:20 dist-test-slave-c12x krb5kdc[17893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222580, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:20.212214 18267 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:35459
I20260808 20:56:20.212261 18472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:35459 every 8 connection(s)
I20260808 20:56:20.212598 18267 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:20.214511 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.204818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:53973 (local address 127.15.232.190:39245)
0808 20:56:20.204894 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.204897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.205105 (+ 208us) server_negotiation.cc:407] Connection header received
0808 20:56:20.205612 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.205613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.205627 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.205711 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:20.206361 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.207205 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.208376 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.208590 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.208675 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.211620 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:20.211621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:20.211621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:20.211628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:20.213268 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.213640 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.213640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.213641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.213694 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.213980 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.213981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.213981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.214091 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:20.214144 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.214385 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.214448 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 20:56:20.214807 18303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.204773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:39245 (local address 127.15.232.130:53973)
0808 20:56:20.204969 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.205012 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.205514 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.205813 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.205816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.206064 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:20.206257 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.206261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.207294 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.207295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.208279 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.208282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.208698 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.208699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.208818 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.209369 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:20.209393 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:20.211506 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:20.213339 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.213344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.213346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.213573 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.213775 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.213776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.213776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.213912 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.214147 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:20.214152 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:20.214272 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.214359 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.214502 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260808 20:56:20.215286 18474 heartbeater.cc:347] Connected to a master server at 127.15.232.190:39245
I20260808 20:56:20.215374 18474 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:20.215595 18474 heartbeater.cc:511] Master 127.15.232.190:39245 requested a full tablet report, sending...
I20260808 20:56:20.216045 17952 ts_manager.cc:194] Registered new tserver with Master: 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459)
I20260808 20:56:20.216637 17952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:53973
I20260808 20:56:20.219451 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 18267
I20260808 20:56:20.219596 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:20.219784 16290 external_mini_cluster.cc:1428] Setting key c8a240dffd06b5f54b27a0332e066f09
I20260808 20:56:20.222375 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.216507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:33889 (local address 127.15.232.190:39245)
0808 20:56:20.216571 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.216572 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.216641 (+ 69us) server_negotiation.cc:407] Connection header received
0808 20:56:20.216725 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.216726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.216737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.216817 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:20.217201 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.217797 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.218501 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.218698 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.218758 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.219703 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:20.219705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:20.219705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:20.219712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:20.221185 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.221538 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.221539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.221540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.221596 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.221795 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.221796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.221797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.221948 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:20.222039 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.222203 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.222282 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 20:56:20.222386 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.216412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:39245 (local address 127.15.232.130:33889)
0808 20:56:20.216583 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.216594 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.216656 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.216825 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.216825 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.217045 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:20.217106 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.217125 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.217883 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.217883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.218396 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.218397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.218766 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.218767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.218826 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.219220 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:20.219224 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:20.219621 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:20.221271 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.221272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.221272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.221448 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.221666 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.221667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.221668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.221728 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.222034 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:20.222035 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:20.222084 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.222176 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.222284 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:20.250293 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:39245
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:20.336138 18491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:20.336387 18491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:20.336431 18491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:20.337626 18491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:20.337697 18491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:20.337733 18491 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 20:56:20.337771 18491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:20.339284 18491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.190:39245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:20.339589 18491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:20.339834 18491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:20.345201 18496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:20.345201 18500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:20.345201 18498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:20.345223 18491 server_base.cc:1034] running on GCE node
I20260808 20:56:20.345783 18491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:20.346063 18491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:20.347239 18491 hybrid_clock.cc:648] HybridClock initialized: now 1786222580347222 us; error 30 us; skew 500 ppm
Aug 08 20:56:20 dist-test-slave-c12x krb5kdc[17893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222580, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:20.350224 18491 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:20.350695 18491 webserver.cc:460] Webserver started at http://127.15.232.131:34923/ using document root <none> and password file <none>
I20260808 20:56:20.350950 18491 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:20.351027 18491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:20.351174 18491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:20.352345 18491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "ff8eb2615e694961a83240b286b6bef5"
format_stamp: "Formatted at 2026-08-08 20:56:20 on dist-test-slave-c12x"
server_key: "a7a9bf1fcfc5c1507fe3e8d2e0fa59cd"
server_key_iv: "f96c9dcc80fbdc444816dfaa8242894a"
server_key_version: "encryptionkey@0"
I20260808 20:56:20.352713 18491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "ff8eb2615e694961a83240b286b6bef5"
format_stamp: "Formatted at 2026-08-08 20:56:20 on dist-test-slave-c12x"
server_key: "a7a9bf1fcfc5c1507fe3e8d2e0fa59cd"
server_key_iv: "f96c9dcc80fbdc444816dfaa8242894a"
server_key_version: "encryptionkey@0"
I20260808 20:56:20.354254 18491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:20.355126 18512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:20.355341 18491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:20.355466 18491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "ff8eb2615e694961a83240b286b6bef5"
format_stamp: "Formatted at 2026-08-08 20:56:20 on dist-test-slave-c12x"
server_key: "a7a9bf1fcfc5c1507fe3e8d2e0fa59cd"
server_key_iv: "f96c9dcc80fbdc444816dfaa8242894a"
server_key_version: "encryptionkey@0"
I20260808 20:56:20.355554 18491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:20.367504 18491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:20.369897 18491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:20.370069 18491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:20.370616 18491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:20.370682 18491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:20.370739 18491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:20.370769 18491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 20:56:20 dist-test-slave-c12x krb5kdc[17893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222580, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:20.378343 18491 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:46131
I20260808 20:56:20.378423 18678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:46131 every 8 connection(s)
I20260808 20:56:20.378726 18491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:20.381397 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.371172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:49755 (local address 127.15.232.190:39245)
0808 20:56:20.371247 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.371251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.371475 (+ 224us) server_negotiation.cc:407] Connection header received
0808 20:56:20.372051 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.372053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.372069 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.372160 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:20.372828 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.373707 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.375502 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.375853 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.375929 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.378284 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:20.378286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:20.378286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:20.378296 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:20.379927 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.380365 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.380366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.380367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.380431 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.380702 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.380703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.380704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.380833 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:20.380907 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.381201 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.381293 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 20:56:20.381583 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.371129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:39245 (local address 127.15.232.131:49755)
0808 20:56:20.371296 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.371339 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.371948 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.372240 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.372244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.372539 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:20.372746 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.372751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.373834 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.373835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.375349 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.375353 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.375979 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.375980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.376118 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.376697 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:20.376705 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:20.378185 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:20.380040 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.380048 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.380050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.380286 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.380514 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.380515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.380515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.380630 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.380899 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:20.380906 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:20.381058 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.381165 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.381338 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260808 20:56:20.382432 18681 heartbeater.cc:347] Connected to a master server at 127.15.232.190:39245
I20260808 20:56:20.382562 18681 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:20.382797 18681 heartbeater.cc:511] Master 127.15.232.190:39245 requested a full tablet report, sending...
I20260808 20:56:20.383244 17951 ts_manager.cc:194] Registered new tserver with Master: ff8eb2615e694961a83240b286b6bef5 (127.15.232.131:46131)
I20260808 20:56:20.383790 17951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:49755
I20260808 20:56:20.386781 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 18491
I20260808 20:56:20.386891 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:20.387075 16290 external_mini_cluster.cc:1428] Setting key 8d839535e5efeb7a55c9c2f8cad073e7
I20260808 20:56:20.388120 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:56:20.390074 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.383569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:39245 (local address 127.15.232.131:53937)
0808 20:56:20.383698 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.383709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.383775 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.383944 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.383945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.384162 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:20.384233 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.384235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.385091 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.385093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.385648 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.385653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.386140 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.386141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.386208 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.386703 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:20.386708 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:20.386998 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:20.389011 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.389012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.389012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.389207 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.389424 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.389424 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.389425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.389481 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.389782 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:20.389783 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:20.389830 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.389896 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.389989 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260808 20:56:20.390179 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.383632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:53937 (local address 127.15.232.190:39245)
0808 20:56:20.383680 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.383682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.383754 (+ 72us) server_negotiation.cc:407] Connection header received
0808 20:56:20.383833 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.383834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.383845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.383913 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:20.384346 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.384909 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.385787 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.386030 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.386134 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.387111 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:20.387112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:20.387113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:20.387122 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:20.388897 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.389286 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.389287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.389287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.389356 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.389541 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.389542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.389542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.389696 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:20.389767 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.389939 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.390033 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 20:56:20 dist-test-slave-c12x krb5kdc[17893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222580, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 20:56:20 dist-test-slave-c12x krb5kdc[17893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222580, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:20.414311 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.407168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.15.232.190:39245)
0808 20:56:20.407259 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.407261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.407271 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:20.407308 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.407309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.407319 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.407378 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:20.408007 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.408525 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.409264 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.409475 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.409543 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.411935 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:20.411937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:20.411937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:20.411944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:20.413155 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.413462 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.413463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.413463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.413514 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.413712 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.413713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.413714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.413840 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:20.413952 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.414088 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.414159 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 20:56:20.416325 17951 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:20.418054 17951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:20.424979 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.420605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:46131 (local address 127.0.0.1:55490)
0808 20:56:20.420955 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.420997 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.421065 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.421377 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.421380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.421382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:20.421456 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.421458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.422785 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.422786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.423955 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.423957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.424634 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.424635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.424730 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.424743 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.424828 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.424884 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":33,"thread_start_us":73,"threads_started":1}
I20260808 20:56:20.425081 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.420612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38529 (local address 127.0.0.1:57654)
0808 20:56:20.421165 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.421177 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.421242 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.421523 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.421524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.421526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:20.421586 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.421588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.422523 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.422525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.423845 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.423848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.424811 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.424812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.424867 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.424870 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.424934 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.424969 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":97,"threads_started":1}
I20260808 20:56:20.425268 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.420990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.15.232.131:46131)
0808 20:56:20.421198 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.421202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.421213 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:20.421257 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.421259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.421297 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.421374 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:20.421546 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.422700 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.424017 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.424540 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.424626 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.424686 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.425121 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.425169 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
I20260808 20:56:20.425299 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.420835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:35459 (local address 127.0.0.1:58766)
0808 20:56:20.421476 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.421486 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.421538 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.421731 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.421732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.421734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:20.421763 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.421764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.422950 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.422951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.423885 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.423906 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.424883 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.424884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.424976 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.424980 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.425035 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.425234 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":17,"thread_start_us":80,"threads_started":1}
I20260808 20:56:20.425563 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.420759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.15.232.129:38529)
0808 20:56:20.420938 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.420943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.421239 (+ 296us) server_negotiation.cc:407] Connection header received
0808 20:56:20.421347 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.421350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.421367 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.421447 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:20.421679 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.422435 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.423946 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.424728 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.424805 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.424850 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.424951 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.424997 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260808 20:56:20.425621 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.420979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.15.232.130:35459)
0808 20:56:20.421169 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.421172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.421599 (+ 427us) server_negotiation.cc:407] Connection header received
0808 20:56:20.421642 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.421644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.421660 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.421732 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:20.421824 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.422857 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.424003 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.424719 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.424794 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.424845 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.425470 (+ 625us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.425518 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
I20260808 20:56:20.425988 18584 tablet_service.cc:1467] Processing CreateTablet for tablet 88866eaf2ae14acbb660e387ee6cac4b (DEFAULT_TABLE table=test-table [id=b80b5c69641a4f2389f4e3fb85782c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:20.426239 18373 tablet_service.cc:1467] Processing CreateTablet for tablet 88866eaf2ae14acbb660e387ee6cac4b (DEFAULT_TABLE table=test-table [id=b80b5c69641a4f2389f4e3fb85782c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:20.426425 18584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88866eaf2ae14acbb660e387ee6cac4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:20.426414 18151 tablet_service.cc:1467] Processing CreateTablet for tablet 88866eaf2ae14acbb660e387ee6cac4b (DEFAULT_TABLE table=test-table [id=b80b5c69641a4f2389f4e3fb85782c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:20.426569 18373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88866eaf2ae14acbb660e387ee6cac4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:20.426694 18151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88866eaf2ae14acbb660e387ee6cac4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:20.429719 18722 tablet_bootstrap.cc:492] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: Bootstrap starting.
I20260808 20:56:20.430486 18723 tablet_bootstrap.cc:492] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: Bootstrap starting.
I20260808 20:56:20.430624 18722 tablet_bootstrap.cc:654] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:20.431021 18722 log.cc:824] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:20.431150 18725 tablet_bootstrap.cc:492] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: Bootstrap starting.
I20260808 20:56:20.431538 18723 tablet_bootstrap.cc:654] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:20.431907 18723 log.cc:824] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:20.432065 18722 tablet_bootstrap.cc:492] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: No bootstrap required, opened a new log
I20260808 20:56:20.432080 18725 tablet_bootstrap.cc:654] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:20.432140 18722 ts_tablet_manager.cc:1397] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:56:20.432379 18725 log.cc:824] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:20.433131 18725 tablet_bootstrap.cc:492] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: No bootstrap required, opened a new log
I20260808 20:56:20.433215 18725 ts_tablet_manager.cc:1397] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:20.434067 18722 raft_consensus.cc:348] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } }
I20260808 20:56:20.434207 18722 raft_consensus.cc:374] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:20.434228 18722 raft_consensus.cc:729] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b02749487f843a0bd0541ef7889d677, State: Initialized, Role: FOLLOWER
I20260808 20:56:20.434362 18722 consensus_queue.cc:260] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } }
I20260808 20:56:20.434620 18722 ts_tablet_manager.cc:1428] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:56:20.434717 18254 heartbeater.cc:503] Master 127.15.232.190:39245 was elected leader, sending a full tablet report...
I20260808 20:56:20.434873 18725 raft_consensus.cc:348] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } }
I20260808 20:56:20.435031 18725 raft_consensus.cc:374] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:20.435086 18725 raft_consensus.cc:729] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1daa4de880d0483aad16e14a395c0e88, State: Initialized, Role: FOLLOWER
I20260808 20:56:20.435206 18725 consensus_queue.cc:260] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } }
I20260808 20:56:20.435307 18723 tablet_bootstrap.cc:492] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: No bootstrap required, opened a new log
I20260808 20:56:20.435369 18723 ts_tablet_manager.cc:1397] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260808 20:56:20.435503 18474 heartbeater.cc:503] Master 127.15.232.190:39245 was elected leader, sending a full tablet report...
I20260808 20:56:20.435508 18725 ts_tablet_manager.cc:1428] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 20:56:20.437067 18723 raft_consensus.cc:348] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } }
I20260808 20:56:20.437215 18723 raft_consensus.cc:374] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:20.437261 18723 raft_consensus.cc:729] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff8eb2615e694961a83240b286b6bef5, State: Initialized, Role: FOLLOWER
I20260808 20:56:20.437372 18723 consensus_queue.cc:260] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } }
I20260808 20:56:20.437631 18723 ts_tablet_manager.cc:1428] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:56:20.437686 18681 heartbeater.cc:503] Master 127.15.232.190:39245 was elected leader, sending a full tablet report...
I20260808 20:56:20.444741 18733 raft_consensus.cc:482] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:20.444852 18733 raft_consensus.cc:504] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } }
I20260808 20:56:20.445181 18733 leader_election.cc:288] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b02749487f843a0bd0541ef7889d677 (127.15.232.129:38529), 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459)
I20260808 20:56:20.448204 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.445503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:47643 (local address 127.15.232.130:35459)
0808 20:56:20.445557 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.445559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.445782 (+ 223us) server_negotiation.cc:407] Connection header received
0808 20:56:20.445818 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.445819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.445827 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.445876 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:20.446009 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.446491 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.447414 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.447826 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.447929 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.447939 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.448094 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.448132 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 20:56:20.448246 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.445427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:39471 (local address 127.15.232.129:38529)
0808 20:56:20.445495 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.445497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.445506 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:20.445535 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.445536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.445546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.445603 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:20.445919 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.446398 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.447363 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.447847 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.447939 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.447948 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.448136 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.448173 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 20:56:20.448297 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.445368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38529 (local address 127.15.232.131:39471)
0808 20:56:20.445413 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.445425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.445465 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.445611 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.445613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.445615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:20.445707 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.445709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.446508 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.446510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.447271 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.447273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.447944 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.447945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.448043 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.448046 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.448124 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.448166 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 20:56:20.448225 18735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.445368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:35459 (local address 127.15.232.131:47643)
0808 20:56:20.445696 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.445707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.445769 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.445881 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.445882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.445883 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:20.445936 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.445937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.446576 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.446577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.447316 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.447318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.447930 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.447931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.448025 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.448028 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.448088 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.448128 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":149,"thread_start_us":110,"threads_started":1}
I20260808 20:56:20.448640 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88866eaf2ae14acbb660e387ee6cac4b" candidate_uuid: "ff8eb2615e694961a83240b286b6bef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b02749487f843a0bd0541ef7889d677" is_pre_election: true
I20260808 20:56:20.448793 18179 raft_consensus.cc:2454] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff8eb2615e694961a83240b286b6bef5 in term 0.
I20260808 20:56:20.448758 18411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88866eaf2ae14acbb660e387ee6cac4b" candidate_uuid: "ff8eb2615e694961a83240b286b6bef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daa4de880d0483aad16e14a395c0e88" is_pre_election: true
I20260808 20:56:20.448897 18411 raft_consensus.cc:2454] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff8eb2615e694961a83240b286b6bef5 in term 0.
I20260808 20:56:20.449002 18513 leader_election.cc:302] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b02749487f843a0bd0541ef7889d677, ff8eb2615e694961a83240b286b6bef5; no voters:
I20260808 20:56:20.449208 18733 raft_consensus.cc:2781] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:20.449252 18733 raft_consensus.cc:482] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:20.449268 18733 raft_consensus.cc:3037] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:20.450227 18733 raft_consensus.cc:504] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } }
I20260808 20:56:20.450385 18733 leader_election.cc:288] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [CANDIDATE]: Term 1 election: Requested vote from peers 2b02749487f843a0bd0541ef7889d677 (127.15.232.129:38529), 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459)
I20260808 20:56:20.450580 18411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88866eaf2ae14acbb660e387ee6cac4b" candidate_uuid: "ff8eb2615e694961a83240b286b6bef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daa4de880d0483aad16e14a395c0e88"
I20260808 20:56:20.450580 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88866eaf2ae14acbb660e387ee6cac4b" candidate_uuid: "ff8eb2615e694961a83240b286b6bef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b02749487f843a0bd0541ef7889d677"
I20260808 20:56:20.450663 18411 raft_consensus.cc:3037] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:20.450663 18179 raft_consensus.cc:3037] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:20.451756 18411 raft_consensus.cc:2454] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff8eb2615e694961a83240b286b6bef5 in term 1.
I20260808 20:56:20.451759 18179 raft_consensus.cc:2454] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff8eb2615e694961a83240b286b6bef5 in term 1.
I20260808 20:56:20.451948 18513 leader_election.cc:302] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b02749487f843a0bd0541ef7889d677, ff8eb2615e694961a83240b286b6bef5; no voters:
I20260808 20:56:20.452065 18733 raft_consensus.cc:2781] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:20.452236 18733 raft_consensus.cc:686] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 1 LEADER]: Becoming Leader. State: Replica: ff8eb2615e694961a83240b286b6bef5, State: Running, Role: LEADER
I20260808 20:56:20.452364 18733 consensus_queue.cc:237] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } }
I20260808 20:56:20.453217 17952 catalog_manager.cc:5495] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff8eb2615e694961a83240b286b6bef5 (127.15.232.131). New cstate: current_term: 1 leader_uuid: "ff8eb2615e694961a83240b286b6bef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } health_report { overall_health: UNKNOWN } } }
W20260808 20:56:20.463133 18475 tablet.cc:2369] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:20.464957 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.461799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.15.232.131:46131)
0808 20:56:20.461879 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.461881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.461913 (+ 32us) server_negotiation.cc:407] Connection header received
0808 20:56:20.461995 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.461996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.462013 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.462068 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:20.462241 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.462735 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.463693 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.463887 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.463945 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.464166 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:20.464279 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:20.464350 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.464387 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.464414 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 20:56:20.467007 18411 raft_consensus.cc:1261] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Refusing update from remote peer ff8eb2615e694961a83240b286b6bef5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:20.467012 18179 raft_consensus.cc:1261] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [term 1 FOLLOWER]: Refusing update from remote peer ff8eb2615e694961a83240b286b6bef5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:20.467494 18737 consensus_queue.cc:1035] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:20.467653 18733 consensus_queue.cc:1035] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:20.470418 18754 mvcc.cc:205] Tried to move back new op lower bound from 7316367689589805056 to 7316367689533677568. Current Snapshot: MvccSnapshot[applied={T|T < 7316367689589805056}]
I20260808 20:56:20.474459 17951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:20.474620 17951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:20.475627 17951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59: Sending DeleteTablet for 3 replicas of tablet 88866eaf2ae14acbb660e387ee6cac4b
I20260808 20:56:20.476162 18373 tablet_service.cc:1514] Processing DeleteTablet for tablet 88866eaf2ae14acbb660e387ee6cac4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:20 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:58766
I20260808 20:56:20.476176 18151 tablet_service.cc:1514] Processing DeleteTablet for tablet 88866eaf2ae14acbb660e387ee6cac4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:20 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:57654
I20260808 20:56:20.476293 17951 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:20.476446 17951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:20.476694 18371 tablet_service.cc:1514] Processing DeleteTablet for tablet 88866eaf2ae14acbb660e387ee6cac4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:20 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:58766
I20260808 20:56:20.476905 18584 tablet_service.cc:1514] Processing DeleteTablet for tablet 88866eaf2ae14acbb660e387ee6cac4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:20 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:55490
I20260808 20:56:20.477167 18770 tablet_replica.cc:330] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: stopping tablet replica
I20260808 20:56:20.477332 18770 raft_consensus.cc:2229] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:20.477492 18769 tablet_replica.cc:330] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: stopping tablet replica
I20260808 20:56:20.477504 18770 raft_consensus.cc:2258] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:20.477618 18769 raft_consensus.cc:2229] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:20.478015 18584 tablet_service.cc:1467] Processing CreateTablet for tablet c227451ad3414e4b84379709792aa3e3 (DEFAULT_TABLE table=test-table [id=aa799a8f620c4f6996cfc842d024a83a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:20.478149 18770 ts_tablet_manager.cc:1905] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:20.478178 18769 raft_consensus.cc:2258] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:20.478367 18151 tablet_service.cc:1467] Processing CreateTablet for tablet c227451ad3414e4b84379709792aa3e3 (DEFAULT_TABLE table=test-table [id=aa799a8f620c4f6996cfc842d024a83a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:20.478504 18151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c227451ad3414e4b84379709792aa3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:20.479086 18769 ts_tablet_manager.cc:1905] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:20.479967 18722 tablet_bootstrap.cc:492] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: Bootstrap starting.
I20260808 20:56:20.480028 18371 tablet_service.cc:1467] Processing CreateTablet for tablet c227451ad3414e4b84379709792aa3e3 (DEFAULT_TABLE table=test-table [id=aa799a8f620c4f6996cfc842d024a83a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:20.480158 18371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c227451ad3414e4b84379709792aa3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:20.480554 18722 tablet_bootstrap.cc:654] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:20.481324 18722 tablet_bootstrap.cc:492] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: No bootstrap required, opened a new log
I20260808 20:56:20.481374 18722 ts_tablet_manager.cc:1397] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:20.481375 18769 ts_tablet_manager.cc:1918] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:20.481474 18769 log.cc:1196] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/88866eaf2ae14acbb660e387ee6cac4b
I20260808 20:56:20.481552 18722 raft_consensus.cc:348] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.481628 18722 raft_consensus.cc:374] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:20.481649 18722 raft_consensus.cc:729] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b02749487f843a0bd0541ef7889d677, State: Initialized, Role: FOLLOWER
I20260808 20:56:20.481695 18722 consensus_queue.cc:260] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.481781 18722 ts_tablet_manager.cc:1428] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:20.482077 18725 tablet_bootstrap.cc:492] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: Bootstrap starting.
I20260808 20:56:20.478169 18584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c227451ad3414e4b84379709792aa3e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:20.482741 18725 tablet_bootstrap.cc:654] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:20.481769 18769 ts_tablet_manager.cc:1939] T 88866eaf2ae14acbb660e387ee6cac4b P 2b02749487f843a0bd0541ef7889d677: Deleting consensus metadata
I20260808 20:56:20.483964 18723 tablet_bootstrap.cc:492] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: Bootstrap starting.
I20260808 20:56:20.484146 18768 tablet_replica.cc:330] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: stopping tablet replica
I20260808 20:56:20.484189 18770 ts_tablet_manager.cc:1918] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:20.484205 17932 catalog_manager.cc:4841] TS 2b02749487f843a0bd0541ef7889d677 (127.15.232.129:38529): tablet 88866eaf2ae14acbb660e387ee6cac4b (table test-table [id=b80b5c69641a4f2389f4e3fb85782c0e]) successfully deleted
I20260808 20:56:20.484306 18768 raft_consensus.cc:2229] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:20.484409 18770 log.cc:1196] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/88866eaf2ae14acbb660e387ee6cac4b
I20260808 20:56:20.484457 18768 raft_consensus.cc:2258] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:20.484426 18723 tablet_bootstrap.cc:654] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:20.484782 18725 tablet_bootstrap.cc:492] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: No bootstrap required, opened a new log
I20260808 20:56:20.484843 18725 ts_tablet_manager.cc:1397] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 20:56:20.484956 18770 ts_tablet_manager.cc:1939] T 88866eaf2ae14acbb660e387ee6cac4b P ff8eb2615e694961a83240b286b6bef5: Deleting consensus metadata
I20260808 20:56:20.484979 18768 ts_tablet_manager.cc:1905] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:20.485013 18725 raft_consensus.cc:348] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.485106 18725 raft_consensus.cc:374] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:20.485136 18725 raft_consensus.cc:729] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1daa4de880d0483aad16e14a395c0e88, State: Initialized, Role: FOLLOWER
I20260808 20:56:20.485189 18725 consensus_queue.cc:260] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.485283 18725 ts_tablet_manager.cc:1428] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:20.485643 17934 catalog_manager.cc:4841] TS ff8eb2615e694961a83240b286b6bef5 (127.15.232.131:46131): tablet 88866eaf2ae14acbb660e387ee6cac4b (table test-table [id=b80b5c69641a4f2389f4e3fb85782c0e]) successfully deleted
I20260808 20:56:20.486452 18768 ts_tablet_manager.cc:1918] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:20.486549 18768 log.cc:1196] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/88866eaf2ae14acbb660e387ee6cac4b
I20260808 20:56:20.486848 18768 ts_tablet_manager.cc:1939] T 88866eaf2ae14acbb660e387ee6cac4b P 1daa4de880d0483aad16e14a395c0e88: Deleting consensus metadata
I20260808 20:56:20.487279 17934 catalog_manager.cc:4841] TS 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459): tablet 88866eaf2ae14acbb660e387ee6cac4b (table test-table [id=b80b5c69641a4f2389f4e3fb85782c0e]) successfully deleted
W20260808 20:56:20.487409 17934 catalog_manager.cc:4808] TS 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459): delete failed for tablet 88866eaf2ae14acbb660e387ee6cac4b because the tablet was not found. No further retry: Not found: Tablet not found: 88866eaf2ae14acbb660e387ee6cac4b
I20260808 20:56:20.487926 18723 tablet_bootstrap.cc:492] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: No bootstrap required, opened a new log
I20260808 20:56:20.487975 18723 ts_tablet_manager.cc:1397] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260808 20:56:20.488193 18723 raft_consensus.cc:348] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.488265 18723 raft_consensus.cc:374] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:20.488279 18723 raft_consensus.cc:729] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff8eb2615e694961a83240b286b6bef5, State: Initialized, Role: FOLLOWER
I20260808 20:56:20.488346 18723 consensus_queue.cc:260] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.488452 18723 ts_tablet_manager.cc:1428] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260808 20:56:20.545382 18255 tablet.cc:2369] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 20:56:20.629351 18683 tablet.cc:2369] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:20.861042 18731 raft_consensus.cc:482] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:20.861173 18731 raft_consensus.cc:504] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.861536 18731 leader_election.cc:288] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459), ff8eb2615e694961a83240b286b6bef5 (127.15.232.131:46131)
I20260808 20:56:20.864825 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.861876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:60007 (local address 127.15.232.130:35459)
0808 20:56:20.861932 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.861934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.862007 (+ 73us) server_negotiation.cc:407] Connection header received
0808 20:56:20.862132 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.862134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.862148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.862189 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:20.862449 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.862977 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.864105 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.864453 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.864515 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.864539 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.864701 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.864750 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 20:56:20.864842 18778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.862035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:46131 (local address 127.15.232.129:34717)
0808 20:56:20.862125 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.862137 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.862211 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.862351 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.862352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.862353 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:20.862424 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.862427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.863056 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.863058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.863987 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.863990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.864577 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.864578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.864638 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.864641 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.864711 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.864752 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":34,"thread_start_us":47,"threads_started":1}
I20260808 20:56:20.864836 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.862116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:34717 (local address 127.15.232.131:46131)
0808 20:56:20.862175 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.862177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.862195 (+ 18us) server_negotiation.cc:407] Connection header received
0808 20:56:20.862265 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.862266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.862278 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.862386 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:20.862499 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.862975 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.864084 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.864466 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.864564 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.864573 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.864728 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.864768 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 20:56:20.864862 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.861790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:35459 (local address 127.15.232.129:60007)
0808 20:56:20.861928 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.861942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.862021 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.862274 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.862275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.862279 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:20.862365 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.862368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.863097 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.863098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.863990 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.863992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.864536 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.864538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.864616 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.864626 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.864674 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.864737 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260808 20:56:20.865360 18411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c227451ad3414e4b84379709792aa3e3" candidate_uuid: "2b02749487f843a0bd0541ef7889d677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daa4de880d0483aad16e14a395c0e88" is_pre_election: true
I20260808 20:56:20.865351 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c227451ad3414e4b84379709792aa3e3" candidate_uuid: "2b02749487f843a0bd0541ef7889d677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff8eb2615e694961a83240b286b6bef5" is_pre_election: true
I20260808 20:56:20.865463 18411 raft_consensus.cc:2454] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b02749487f843a0bd0541ef7889d677 in term 0.
I20260808 20:56:20.865513 18613 raft_consensus.cc:2454] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b02749487f843a0bd0541ef7889d677 in term 0.
I20260808 20:56:20.865751 18082 leader_election.cc:302] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1daa4de880d0483aad16e14a395c0e88, 2b02749487f843a0bd0541ef7889d677; no voters:
I20260808 20:56:20.865875 18731 raft_consensus.cc:2781] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:20.865918 18731 raft_consensus.cc:482] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:20.865947 18731 raft_consensus.cc:3037] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:20.866653 18731 raft_consensus.cc:504] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.866798 18731 leader_election.cc:288] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [CANDIDATE]: Term 1 election: Requested vote from peers 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459), ff8eb2615e694961a83240b286b6bef5 (127.15.232.131:46131)
I20260808 20:56:20.866993 18411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c227451ad3414e4b84379709792aa3e3" candidate_uuid: "2b02749487f843a0bd0541ef7889d677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daa4de880d0483aad16e14a395c0e88"
I20260808 20:56:20.867008 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c227451ad3414e4b84379709792aa3e3" candidate_uuid: "2b02749487f843a0bd0541ef7889d677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff8eb2615e694961a83240b286b6bef5"
I20260808 20:56:20.867082 18613 raft_consensus.cc:3037] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:20.867082 18411 raft_consensus.cc:3037] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:20.867875 18411 raft_consensus.cc:2454] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b02749487f843a0bd0541ef7889d677 in term 1.
I20260808 20:56:20.867937 18613 raft_consensus.cc:2454] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b02749487f843a0bd0541ef7889d677 in term 1.
I20260808 20:56:20.868095 18082 leader_election.cc:302] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b02749487f843a0bd0541ef7889d677, ff8eb2615e694961a83240b286b6bef5; no voters:
I20260808 20:56:20.868201 18731 raft_consensus.cc:2781] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:20.868314 18731 raft_consensus.cc:686] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 1 LEADER]: Becoming Leader. State: Replica: 2b02749487f843a0bd0541ef7889d677, State: Running, Role: LEADER
I20260808 20:56:20.868371 18731 consensus_queue.cc:237] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.869091 17951 catalog_manager.cc:5495] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b02749487f843a0bd0541ef7889d677 (127.15.232.129). New cstate: current_term: 1 leader_uuid: "2b02749487f843a0bd0541ef7889d677" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } health_report { overall_health: UNKNOWN } } }
Aug 08 20:56:20 dist-test-slave-c12x krb5kdc[17893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222579, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:20.948235 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.940281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:39245 (local address 127.0.0.1:41406)
0808 20:56:20.940496 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:20.940511 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:20.940616 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:20.940775 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:20.940776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:20.941139 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:20.941185 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.941188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.941859 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.941860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.942447 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:20.942449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.942829 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.942830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:20.942883 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.943485 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:20.943490 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:20.945174 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:20.946942 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.946950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.946953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.947186 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.947394 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:20.947394 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:20.947395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:20.947534 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:20.947810 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:20.947817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:20.947867 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 20:56:20.947934 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 20:56:20.948043 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":70,"threads_started":1}
I20260808 20:56:20.948240 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:20.940349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.15.232.190:39245)
0808 20:56:20.940586 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:20.940588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:20.940599 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:20.940684 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:20.940685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:20.940701 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:20.940772 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:20.941279 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.941781 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.942546 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:20.942735 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:20.942816 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:20.945272 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:20.945273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:20.945274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:20.945280 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:20.946832 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.947267 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.947268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.947269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.947327 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:20.947598 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:20.947599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:20.947599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:20.947732 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:20.947791 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:20.948000 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 20:56:20.948130 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":218,"thread_start_us":53,"threads_started":1}
I20260808 20:56:20.950735 17951 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:41406:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 20:56:20.951088 17951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:20.952742 18371 tablet_service.cc:1467] Processing CreateTablet for tablet 058a5a959d834417a3b4fd8274be1c51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e0f79b49aa9e4e6ca64fecaeaf8b6b9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 20:56:20.952746 18151 tablet_service.cc:1467] Processing CreateTablet for tablet 058a5a959d834417a3b4fd8274be1c51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e0f79b49aa9e4e6ca64fecaeaf8b6b9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 20:56:20.952855 18584 tablet_service.cc:1467] Processing CreateTablet for tablet 058a5a959d834417a3b4fd8274be1c51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e0f79b49aa9e4e6ca64fecaeaf8b6b9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 20:56:20.952979 18371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058a5a959d834417a3b4fd8274be1c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:20.953027 18151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058a5a959d834417a3b4fd8274be1c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:20.953047 18584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 058a5a959d834417a3b4fd8274be1c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:20.955762 18725 tablet_bootstrap.cc:492] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: Bootstrap starting.
I20260808 20:56:20.955802 18723 tablet_bootstrap.cc:492] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: Bootstrap starting.
I20260808 20:56:20.956012 18722 tablet_bootstrap.cc:492] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: Bootstrap starting.
I20260808 20:56:20.956291 18723 tablet_bootstrap.cc:654] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:20.956287 18725 tablet_bootstrap.cc:654] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:20.956575 18722 tablet_bootstrap.cc:654] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:20.956920 18723 tablet_bootstrap.cc:492] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: No bootstrap required, opened a new log
I20260808 20:56:20.957000 18723 ts_tablet_manager.cc:1397] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260808 20:56:20.957123 18723 raft_consensus.cc:348] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.957206 18723 raft_consensus.cc:374] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:20.957229 18723 raft_consensus.cc:729] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff8eb2615e694961a83240b286b6bef5, State: Initialized, Role: FOLLOWER
I20260808 20:56:20.957239 18722 tablet_bootstrap.cc:492] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: No bootstrap required, opened a new log
I20260808 20:56:20.957284 18722 ts_tablet_manager.cc:1397] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:20.957285 18723 consensus_queue.cc:260] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.957404 18723 ts_tablet_manager.cc:1428] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:20.957451 18722 raft_consensus.cc:348] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.957535 18722 raft_consensus.cc:374] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:20.957458 18765 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:20.957556 18722 raft_consensus.cc:729] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b02749487f843a0bd0541ef7889d677, State: Initialized, Role: FOLLOWER
I20260808 20:56:20.957574 18765 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:20.957602 18722 consensus_queue.cc:260] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.957691 18722 ts_tablet_manager.cc:1428] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:20.957816 18731 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:20.957927 18731 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:20.958011 18725 tablet_bootstrap.cc:492] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: No bootstrap required, opened a new log
I20260808 20:56:20.958067 18725 ts_tablet_manager.cc:1397] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:20.958250 18725 raft_consensus.cc:348] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.958338 18725 raft_consensus.cc:374] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:20.958364 18725 raft_consensus.cc:729] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1daa4de880d0483aad16e14a395c0e88, State: Initialized, Role: FOLLOWER
I20260808 20:56:20.958410 18725 consensus_queue.cc:260] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:20.958524 18725 ts_tablet_manager.cc:1428] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:20.958671 18732 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:20.958757 18732 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:21.175239 18765 raft_consensus.cc:482] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:21.175344 18765 raft_consensus.cc:504] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:21.175555 18765 leader_election.cc:288] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b02749487f843a0bd0541ef7889d677 (127.15.232.129:38529), 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459)
I20260808 20:56:21.175868 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058a5a959d834417a3b4fd8274be1c51" candidate_uuid: "ff8eb2615e694961a83240b286b6bef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b02749487f843a0bd0541ef7889d677" is_pre_election: true
I20260808 20:56:21.175957 18179 raft_consensus.cc:2454] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff8eb2615e694961a83240b286b6bef5 in term 0.
I20260808 20:56:21.175879 18411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058a5a959d834417a3b4fd8274be1c51" candidate_uuid: "ff8eb2615e694961a83240b286b6bef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daa4de880d0483aad16e14a395c0e88" is_pre_election: true
I20260808 20:56:21.176033 18411 raft_consensus.cc:2454] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff8eb2615e694961a83240b286b6bef5 in term 0.
I20260808 20:56:21.176136 18513 leader_election.cc:302] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b02749487f843a0bd0541ef7889d677, ff8eb2615e694961a83240b286b6bef5; no voters:
I20260808 20:56:21.176251 18765 raft_consensus.cc:2781] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:21.176304 18765 raft_consensus.cc:482] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:21.176332 18765 raft_consensus.cc:3037] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:21.177152 18765 raft_consensus.cc:504] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:21.177306 18765 leader_election.cc:288] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [CANDIDATE]: Term 1 election: Requested vote from peers 2b02749487f843a0bd0541ef7889d677 (127.15.232.129:38529), 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459)
I20260808 20:56:21.177493 18411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058a5a959d834417a3b4fd8274be1c51" candidate_uuid: "ff8eb2615e694961a83240b286b6bef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1daa4de880d0483aad16e14a395c0e88"
I20260808 20:56:21.177506 18179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "058a5a959d834417a3b4fd8274be1c51" candidate_uuid: "ff8eb2615e694961a83240b286b6bef5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b02749487f843a0bd0541ef7889d677"
I20260808 20:56:21.177572 18411 raft_consensus.cc:3037] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:21.177599 18179 raft_consensus.cc:3037] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:21.178210 18411 raft_consensus.cc:2454] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff8eb2615e694961a83240b286b6bef5 in term 1.
I20260808 20:56:21.178335 18179 raft_consensus.cc:2454] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff8eb2615e694961a83240b286b6bef5 in term 1.
I20260808 20:56:21.178401 18519 leader_election.cc:302] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1daa4de880d0483aad16e14a395c0e88, ff8eb2615e694961a83240b286b6bef5; no voters:
I20260808 20:56:21.178499 18765 raft_consensus.cc:2781] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:21.178623 18765 raft_consensus.cc:686] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [term 1 LEADER]: Becoming Leader. State: Replica: ff8eb2615e694961a83240b286b6bef5, State: Running, Role: LEADER
I20260808 20:56:21.178710 18765 consensus_queue.cc:237] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } }
I20260808 20:56:21.178993 18795 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: TxnStatusTablet state changed. Reason: New leader ff8eb2615e694961a83240b286b6bef5. Latest consensus state: current_term: 1 leader_uuid: "ff8eb2615e694961a83240b286b6bef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:21.179098 18795 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: This TxnStatusTablet replica's current role is: LEADER
I20260808 20:56:21.179322 18797 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 20:56:21.179355 17951 catalog_manager.cc:5495] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff8eb2615e694961a83240b286b6bef5 (127.15.232.131). New cstate: current_term: 1 leader_uuid: "ff8eb2615e694961a83240b286b6bef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } health_report { overall_health: HEALTHY } } }
I20260808 20:56:21.179481 18797 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 20:56:21.186761 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:21.184313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.15.232.131:46131)
0808 20:56:21.184358 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:21.184361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:21.184401 (+ 40us) server_negotiation.cc:407] Connection header received
0808 20:56:21.184494 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:21.184495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:21.184506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:21.184563 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:21.184688 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.185175 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.185972 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.186179 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.186267 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:21.186460 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:21.186503 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:21.186574 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:21.186660 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 20:56:21.186683 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 20:56:21.186779 18788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:21.184262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:46131 (local address 127.0.0.1:55514)
0808 20:56:21.184357 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:21.184369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:21.184431 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:21.184571 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:21.184571 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:21.184572 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:21.184625 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:21.184628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.185260 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.185261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:21.185864 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:21.185865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.186269 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.186271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:21.186335 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:21.186359 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:21.186577 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:21.186589 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 20:56:21.186654 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 20:56:21.186710 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 20:56:21.188087 18411 raft_consensus.cc:1261] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Refusing update from remote peer ff8eb2615e694961a83240b286b6bef5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:21.188129 18179 raft_consensus.cc:1261] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677 [term 1 FOLLOWER]: Refusing update from remote peer ff8eb2615e694961a83240b286b6bef5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:21.188326 18795 consensus_queue.cc:1035] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:21.188429 18765 consensus_queue.cc:1035] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:21.188957 18731 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: TxnStatusTablet state changed. Reason: New leader ff8eb2615e694961a83240b286b6bef5. Latest consensus state: current_term: 1 leader_uuid: "ff8eb2615e694961a83240b286b6bef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:21.189052 18731 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:21.189476 18732 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: TxnStatusTablet state changed. Reason: New leader ff8eb2615e694961a83240b286b6bef5. Latest consensus state: current_term: 1 leader_uuid: "ff8eb2615e694961a83240b286b6bef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:21.189594 18732 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:21.189967 18765 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff8eb2615e694961a83240b286b6bef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:21.190063 18765 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: This TxnStatusTablet replica's current role is: LEADER
I20260808 20:56:21.190271 18731 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff8eb2615e694961a83240b286b6bef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:21.190351 18731 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P 2b02749487f843a0bd0541ef7889d677: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:21.191462 18795 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff8eb2615e694961a83240b286b6bef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:21.191568 18795 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P ff8eb2615e694961a83240b286b6bef5: This TxnStatusTablet replica's current role is: LEADER
I20260808 20:56:21.191784 18806 mvcc.cc:205] Tried to move back new op lower bound from 7316367692544614400 to 7316367692508360704. Current Snapshot: MvccSnapshot[applied={T|T < 7316367692544614400}]
I20260808 20:56:21.191879 18732 tablet_replica.cc:439] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff8eb2615e694961a83240b286b6bef5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b02749487f843a0bd0541ef7889d677" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38529 } } peers { permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 } } peers { permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 } } }
I20260808 20:56:21.191984 18732 tablet_replica.cc:442] T 058a5a959d834417a3b4fd8274be1c51 P 1daa4de880d0483aad16e14a395c0e88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:21.195679 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:21.192627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.15.232.129:38529)
0808 20:56:21.192735 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:21.192738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:21.192747 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:21.192832 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:21.192833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:21.192855 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:21.192940 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:21.193141 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.193957 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.194726 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.195057 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.195121 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:21.195244 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:21.195378 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:21.195505 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:21.195563 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 20:56:21.195586 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":68,"threads_started":1}
I20260808 20:56:21.201372 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:21.198349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:46131 (local address 127.15.232.129:34041)
0808 20:56:21.198541 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:21.198553 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:21.198633 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:21.198813 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:21.198814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:21.198815 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:21.198861 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:21.198863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.199596 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.199597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:21.200411 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:21.200414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.200859 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.200860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:21.200922 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:21.200953 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:21.201171 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:21.201175 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:21.201234 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 20:56:21.201278 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260808 20:56:21.201349 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:21.198451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:34041 (local address 127.15.232.131:46131)
0808 20:56:21.198513 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:21.198515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:21.198615 (+ 100us) server_negotiation.cc:407] Connection header received
0808 20:56:21.198735 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:21.198736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:21.198751 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:21.198808 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:21.198954 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.199509 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.200529 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.200764 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.200855 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:21.201060 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:21.201097 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:21.201171 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:21.201239 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 20:56:21.201262 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 20:56:21.204617 18411 raft_consensus.cc:1261] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Refusing update from remote peer 2b02749487f843a0bd0541ef7889d677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:21.204721 18613 raft_consensus.cc:1261] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [term 1 FOLLOWER]: Refusing update from remote peer 2b02749487f843a0bd0541ef7889d677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:21.204877 18731 consensus_queue.cc:1035] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1daa4de880d0483aad16e14a395c0e88" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 35459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:21.205010 18779 consensus_queue.cc:1035] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff8eb2615e694961a83240b286b6bef5" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 46131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:21.215319 18837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:21.211577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38529 (local address 127.15.232.131:58733)
0808 20:56:21.211741 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:21.211752 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:21.211802 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:21.212019 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:21.212020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:21.212022 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:21.212077 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:21.212079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.213667 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.213668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:21.214448 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:21.214451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.214857 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.214858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:21.214916 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:21.214938 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:21.215150 (+ 212us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:21.215152 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 20:56:21.215195 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 20:56:21.215243 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260808 20:56:21.215301 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:21.211668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:58733 (local address 127.15.232.129:38529)
0808 20:56:21.211792 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:21.211795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:21.211806 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:21.211858 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:21.211859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:21.211871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:21.211957 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:21.212252 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.213581 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.214545 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.214766 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.214844 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:21.215020 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:21.215076 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:21.215133 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:21.215198 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 20:56:21.215222 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 20:56:21.225315 17951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:21.225455 17951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:21.226205 17951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3984ab3e4c6f443eab0496f35f0d9a59: Sending DeleteTablet for 3 replicas of tablet c227451ad3414e4b84379709792aa3e3
I20260808 20:56:21.226507 18151 tablet_service.cc:1514] Processing DeleteTablet for tablet c227451ad3414e4b84379709792aa3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:21 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:57654
I20260808 20:56:21.226572 18371 tablet_service.cc:1514] Processing DeleteTablet for tablet c227451ad3414e4b84379709792aa3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:21 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:58766
I20260808 20:56:21.227413 18584 tablet_service.cc:1514] Processing DeleteTablet for tablet c227451ad3414e4b84379709792aa3e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:21 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:55490
I20260808 20:56:21.231564 18847 tablet_replica.cc:330] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: stopping tablet replica
I20260808 20:56:21.231699 18847 raft_consensus.cc:2229] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:21.231812 18847 raft_consensus.cc:2258] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:21.232235 18847 ts_tablet_manager.cc:1905] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 20:56:21 dist-test-slave-c12x krb5kdc[17893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222580, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:21.234817 18847 ts_tablet_manager.cc:1918] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:21.234990 18847 log.cc:1196] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/c227451ad3414e4b84379709792aa3e3
I20260808 20:56:21.235734 18847 ts_tablet_manager.cc:1939] T c227451ad3414e4b84379709792aa3e3 P ff8eb2615e694961a83240b286b6bef5: Deleting consensus metadata
I20260808 20:56:21.236215 17934 catalog_manager.cc:4841] TS ff8eb2615e694961a83240b286b6bef5 (127.15.232.131:46131): tablet c227451ad3414e4b84379709792aa3e3 (table test-table [id=aa799a8f620c4f6996cfc842d024a83a]) successfully deleted
I20260808 20:56:21.237545 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:21.227523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57676 (local address 127.15.232.129:38529)
0808 20:56:21.227566 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:21.227568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:21.227957 (+ 389us) server_negotiation.cc:407] Connection header received
0808 20:56:21.228032 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:21.228033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:21.228045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:21.228091 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:21.228660 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.229156 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.229869 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.230082 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.230167 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:21.233909 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:21.233918 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:21.233919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:21.233927 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:21.236305 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:21.236672 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:21.236676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:21.236676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:21.236748 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:21.236957 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:21.236958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:21.236959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:21.237118 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:21.237285 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:21.237326 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 20:56:21.237411 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 20:56:21.237937 18245 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57676
I20260808 20:56:21.238286 18842 tablet_replica.cc:330] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: stopping tablet replica
I20260808 20:56:21.238376 18842 raft_consensus.cc:2229] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:21.238430 18842 raft_consensus.cc:2258] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:21.238826 18842 ts_tablet_manager.cc:1905] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:21.239625 18841 tablet_replica.cc:330] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: stopping tablet replica
I20260808 20:56:21.239818 18841 raft_consensus.cc:2229] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:21.239996 18841 raft_consensus.cc:2258] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:21.240393 18841 ts_tablet_manager.cc:1905] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:21.241212 18842 ts_tablet_manager.cc:1918] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:21.241291 18842 log.cc:1196] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/c227451ad3414e4b84379709792aa3e3
I20260808 20:56:21.241535 18842 ts_tablet_manager.cc:1939] T c227451ad3414e4b84379709792aa3e3 P 1daa4de880d0483aad16e14a395c0e88: Deleting consensus metadata
I20260808 20:56:21.241902 17934 catalog_manager.cc:4841] TS 1daa4de880d0483aad16e14a395c0e88 (127.15.232.130:35459): tablet c227451ad3414e4b84379709792aa3e3 (table test-table [id=aa799a8f620c4f6996cfc842d024a83a]) successfully deleted
I20260808 20:56:21.242288 18841 ts_tablet_manager.cc:1918] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:21.242354 18841 log.cc:1196] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/c227451ad3414e4b84379709792aa3e3
I20260808 20:56:21.242599 18841 ts_tablet_manager.cc:1939] T c227451ad3414e4b84379709792aa3e3 P 2b02749487f843a0bd0541ef7889d677: Deleting consensus metadata
I20260808 20:56:21.243177 17932 catalog_manager.cc:4841] TS 2b02749487f843a0bd0541ef7889d677 (127.15.232.129:38529): tablet c227451ad3414e4b84379709792aa3e3 (table test-table [id=aa799a8f620c4f6996cfc842d024a83a]) successfully deleted
I20260808 20:56:21.247799 18789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:21.239187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.15.232.190:39245)
0808 20:56:21.239238 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:21.239241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:21.239415 (+ 174us) server_negotiation.cc:407] Connection header received
0808 20:56:21.239548 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:21.239549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:21.239564 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:21.239626 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:21.240475 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.241293 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.242334 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:21.242659 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:21.242707 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:21.244415 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:21.244417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:21.244417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:21.244425 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:21.246326 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:21.246748 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:21.246749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:21.246750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:21.246808 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:21.247006 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:21.247006 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:21.247007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:21.247144 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:21.247221 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:21.247491 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 20:56:21.247693 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 20:56:21.248704 17951 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41410
I20260808 20:56:21.249676 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 18052
I20260808 20:56:21.277781 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 18267
I20260808 20:56:21.293975 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 18491
I20260808 20:56:21.308733 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 17910
2026-08-08T20:56:21Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3057 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:21 dist-test-slave-c12x krb5kdc[18856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:21 dist-test-slave-c12x krb5kdc[18856](info): set up 2 sockets
Aug 08 20:56:21 dist-test-slave-c12x krb5kdc[18856](info): commencing operation
krb5kdc: starting...
W20260808 20:56:22.542945 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:22 dist-test-slave-c12x krb5kdc[18856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:22.674854 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:42047
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:42047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:22.762930 18873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:22.763211 18873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:22.763252 18873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:22.764938 18873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:22.765036 18873 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 20:56:22.765062 18873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:22.765089 18873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:22.765116 18873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:22.765148 18873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:22.767145 18873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:42047
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:42047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:22.767506 18873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:22.767779 18873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:22.773139 18879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:22.773095 18883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:22.773209 18873 server_base.cc:1034] running on GCE node
W20260808 20:56:22.773095 18881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:22.773536 18873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:22.773830 18873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:22.774995 18873 hybrid_clock.cc:648] HybridClock initialized: now 1786222582774979 us; error 32 us; skew 500 ppm
Aug 08 20:56:22 dist-test-slave-c12x krb5kdc[18856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222582, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:22.777930 18873 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:22.778414 18873 webserver.cc:460] Webserver started at http://127.15.232.190:36303/ using document root <none> and password file <none>
I20260808 20:56:22.778689 18873 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:22.778740 18873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:22.778875 18873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:22.780088 18873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "bddb3b47089244ad941421fde55eabe7"
format_stamp: "Formatted at 2026-08-08 20:56:22 on dist-test-slave-c12x"
server_key: "41b17334d34aee51fe4edcfff5dd5407"
server_key_iv: "647a4260c2ddebfbab04829577c7db9a"
server_key_version: "encryptionkey@0"
I20260808 20:56:22.780473 18873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "bddb3b47089244ad941421fde55eabe7"
format_stamp: "Formatted at 2026-08-08 20:56:22 on dist-test-slave-c12x"
server_key: "41b17334d34aee51fe4edcfff5dd5407"
server_key_iv: "647a4260c2ddebfbab04829577c7db9a"
server_key_version: "encryptionkey@0"
I20260808 20:56:22.782079 18873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:22.782897 18895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:22.783128 18873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:22.783239 18873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "bddb3b47089244ad941421fde55eabe7"
format_stamp: "Formatted at 2026-08-08 20:56:22 on dist-test-slave-c12x"
server_key: "41b17334d34aee51fe4edcfff5dd5407"
server_key_iv: "647a4260c2ddebfbab04829577c7db9a"
server_key_version: "encryptionkey@0"
I20260808 20:56:22.783316 18873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:22.801964 18873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:22.804503 18873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:22.804697 18873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:22.810213 18873 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:42047
I20260808 20:56:22.810353 18990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:42047 every 8 connection(s)
I20260808 20:56:22.810793 18873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:22.811676 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 18873
I20260808 20:56:22.811857 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:22.811800 18992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:22.812086 16290 external_mini_cluster.cc:1428] Setting key 6b9b591ef960c47bd464f6d5dff77e2d
I20260808 20:56:22.815696 18992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7: Bootstrap starting.
I20260808 20:56:22.816447 18992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:22.816753 18992 log.cc:824] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:22.817665 18992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7: No bootstrap required, opened a new log
Aug 08 20:56:22 dist-test-slave-c12x krb5kdc[18856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:22.820282 18992 raft_consensus.cc:348] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddb3b47089244ad941421fde55eabe7" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42047 } }
I20260808 20:56:22.820484 18992 raft_consensus.cc:374] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:22.820528 18992 raft_consensus.cc:729] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bddb3b47089244ad941421fde55eabe7, State: Initialized, Role: FOLLOWER
I20260808 20:56:22.820691 18992 consensus_queue.cc:260] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddb3b47089244ad941421fde55eabe7" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42047 } }
I20260808 20:56:22.820796 18992 raft_consensus.cc:388] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:22.820858 18992 raft_consensus.cc:482] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:22.820920 18992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:22.821827 18992 raft_consensus.cc:504] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddb3b47089244ad941421fde55eabe7" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42047 } }
I20260808 20:56:22.821969 18992 leader_election.cc:302] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bddb3b47089244ad941421fde55eabe7; no voters:
I20260808 20:56:22.822180 18992 leader_election.cc:288] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:22.822255 18998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:22.822459 18998 raft_consensus.cc:686] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [term 1 LEADER]: Becoming Leader. State: Replica: bddb3b47089244ad941421fde55eabe7, State: Running, Role: LEADER
I20260808 20:56:22.822475 18992 sys_catalog.cc:564] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:22.822566 18998 consensus_queue.cc:237] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddb3b47089244ad941421fde55eabe7" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42047 } }
I20260808 20:56:22.823243 18999 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bddb3b47089244ad941421fde55eabe7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddb3b47089244ad941421fde55eabe7" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42047 } } }
I20260808 20:56:22.823251 19000 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bddb3b47089244ad941421fde55eabe7. Latest consensus state: current_term: 1 leader_uuid: "bddb3b47089244ad941421fde55eabe7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddb3b47089244ad941421fde55eabe7" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42047 } } }
I20260808 20:56:22.823479 19000 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:22.823460 18999 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:22.824132 19012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:22.824750 19012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:22.824960 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:22.812846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.15.232.190:42047)
0808 20:56:22.813066 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:22.813077 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:22.813111 (+ 34us) server_negotiation.cc:407] Connection header received
0808 20:56:22.813672 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:22.813683 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:22.813754 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:22.813920 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:22.814541 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:22.815997 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:22.816999 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:22.817559 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:22.817686 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:22.820498 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:22.820503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:22.820503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:22.820518 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:22.822646 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:22.823182 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:22.823185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:22.823186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:22.823308 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:22.823579 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:22.823580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:22.823580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:22.823918 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:22.824097 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:22.824574 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 20:56:22.824724 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":95,"threads_started":1}
I20260808 20:56:22.826499 19012 catalog_manager.cc:1347] Generated new cluster ID: 7b34be7d542349c6b3c77177f490db02
I20260808 20:56:22.826565 19012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:22.848768 19012 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:22.849599 19012 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:22.854997 19012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7: Generated new TSK 0
I20260808 20:56:22.855225 19012 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:22.886641 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:42047
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:22.982246 19030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:22.982491 19030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:22.982530 19030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:22.983826 19030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:22.983873 19030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:22.983896 19030 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 20:56:22.983909 19030 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 20:56:22.983927 19030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:22.985531 19030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.190:42047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:22.985745 19030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:22.985944 19030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:22.991308 19039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:22.991325 19041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:22.991302 19038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:22.991570 19030 server_base.cc:1034] running on GCE node
I20260808 20:56:22.991823 19030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:22.992048 19030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:22.993211 19030 hybrid_clock.cc:648] HybridClock initialized: now 1786222582993195 us; error 31 us; skew 500 ppm
Aug 08 20:56:22 dist-test-slave-c12x krb5kdc[18856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222582, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:22.996222 19030 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:22.996870 19030 webserver.cc:460] Webserver started at http://127.15.232.129:34523/ using document root <none> and password file <none>
I20260808 20:56:22.997140 19030 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:22.997215 19030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:22.997367 19030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:22.998605 19030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "7be5243b99494a19b523e2157a1118a8"
format_stamp: "Formatted at 2026-08-08 20:56:22 on dist-test-slave-c12x"
server_key: "0517f370e5da8c7c96f71ab354e4786d"
server_key_iv: "6a3d025f52643b8e2b24a8e688d54775"
server_key_version: "encryptionkey@0"
I20260808 20:56:22.999114 19030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "7be5243b99494a19b523e2157a1118a8"
format_stamp: "Formatted at 2026-08-08 20:56:22 on dist-test-slave-c12x"
server_key: "0517f370e5da8c7c96f71ab354e4786d"
server_key_iv: "6a3d025f52643b8e2b24a8e688d54775"
server_key_version: "encryptionkey@0"
I20260808 20:56:23.000921 19030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:23.001892 19053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.002228 19030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:23.002364 19030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "7be5243b99494a19b523e2157a1118a8"
format_stamp: "Formatted at 2026-08-08 20:56:22 on dist-test-slave-c12x"
server_key: "0517f370e5da8c7c96f71ab354e4786d"
server_key_iv: "6a3d025f52643b8e2b24a8e688d54775"
server_key_version: "encryptionkey@0"
I20260808 20:56:23.002482 19030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:23.016353 19030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:23.018774 19030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:23.018946 19030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:23.019550 19030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:23.019605 19030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.019649 19030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:23.019675 19030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.027166 19030 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:38947
I20260808 20:56:23.027249 19225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:38947 every 8 connection(s)
Aug 08 20:56:23 dist-test-slave-c12x krb5kdc[18856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222582, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:23.027657 19030 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:23.030654 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.020261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:51233 (local address 127.15.232.190:42047)
0808 20:56:23.020316 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.020318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.020590 (+ 272us) server_negotiation.cc:407] Connection header received
0808 20:56:23.021294 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.021296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.021315 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.021436 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:23.022423 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.023383 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.024930 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.025293 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.025392 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.028073 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:23.028075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:23.028075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:23.028085 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:23.029329 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.029739 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.029741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.029741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.029804 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.030041 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.030042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.030042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.030162 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:23.030215 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.030482 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.030584 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:56:23.030787 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.020230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:42047 (local address 127.15.232.129:51233)
0808 20:56:23.020454 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.020493 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.021111 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.021581 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.021586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.021997 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:23.022300 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.022308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.023541 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.023542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.024828 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.024831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.025414 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.025415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.025538 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.026235 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:23.026244 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:23.027926 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:23.029409 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.029416 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.029417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.029649 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.029875 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.029876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.029876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.029983 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.030204 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:23.030207 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:23.030322 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.030408 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.030577 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":97,"threads_started":1}
I20260808 20:56:23.031643 19226 heartbeater.cc:347] Connected to a master server at 127.15.232.190:42047
I20260808 20:56:23.031759 19226 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:23.031991 19226 heartbeater.cc:511] Master 127.15.232.190:42047 requested a full tablet report, sending...
I20260808 20:56:23.032624 18922 ts_manager.cc:194] Registered new tserver with Master: 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947)
I20260808 20:56:23.033658 18922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:51233
I20260808 20:56:23.033638 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 19030
I20260808 20:56:23.033828 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:23.034073 16290 external_mini_cluster.cc:1428] Setting key 2f3dd95acff0a656bcdd30997ece5247
I20260808 20:56:23.040306 19234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.032891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:42047 (local address 127.15.232.129:48403)
0808 20:56:23.033085 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.033096 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.033161 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.033287 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.033288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.033513 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:23.033641 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.033644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.035115 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.035117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.035916 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.035919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.036453 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.036454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.036513 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.037085 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:23.037091 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:23.037521 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:23.039183 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.039184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.039185 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.039369 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.039630 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.039630 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.039631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.039695 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.039959 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:23.039960 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:23.040008 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.040089 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.040203 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":96,"threads_started":1}
I20260808 20:56:23.040303 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.032968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:48403 (local address 127.15.232.190:42047)
0808 20:56:23.033012 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.033014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.033160 (+ 146us) server_negotiation.cc:407] Connection header received
0808 20:56:23.033221 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.033222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.033231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.033274 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:23.033938 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.034851 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.036000 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.036307 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.036386 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.037637 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:23.037638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:23.037639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:23.037647 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:23.039065 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.039491 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.039493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.039493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.039556 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.039766 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.039767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.039767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.039888 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:23.039955 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.040116 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.040202 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":210,"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:23.067847 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:42047
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:23.155340 19240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:23.155635 19240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:23.155664 19240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:23.156816 19240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:23.156855 19240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:23.156875 19240 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 20:56:23.156888 19240 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 20:56:23.156904 19240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:23.158375 19240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.190:42047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:23.158598 19240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:23.158782 19240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:23.164130 19245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:23.164225 19246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:23.164130 19250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:23.164534 19240 server_base.cc:1034] running on GCE node
I20260808 20:56:23.164757 19240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:23.165028 19240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:23.166220 19240 hybrid_clock.cc:648] HybridClock initialized: now 1786222583166202 us; error 49 us; skew 500 ppm
Aug 08 20:56:23 dist-test-slave-c12x krb5kdc[18856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:23.169409 19240 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:23.169871 19240 webserver.cc:460] Webserver started at http://127.15.232.130:41289/ using document root <none> and password file <none>
I20260808 20:56:23.170110 19240 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:23.170159 19240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:23.170312 19240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:23.171665 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "12ff3ef0cf31433a928e08b40d506aac"
format_stamp: "Formatted at 2026-08-08 20:56:23 on dist-test-slave-c12x"
server_key: "2959594663823bb2d31304eb177cbb91"
server_key_iv: "95c2883678c29faa982a16e0cfc758cc"
server_key_version: "encryptionkey@0"
I20260808 20:56:23.172082 19240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "12ff3ef0cf31433a928e08b40d506aac"
format_stamp: "Formatted at 2026-08-08 20:56:23 on dist-test-slave-c12x"
server_key: "2959594663823bb2d31304eb177cbb91"
server_key_iv: "95c2883678c29faa982a16e0cfc758cc"
server_key_version: "encryptionkey@0"
I20260808 20:56:23.173542 19240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 20:56:23.174387 19262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.174671 19240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:23.174774 19240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "12ff3ef0cf31433a928e08b40d506aac"
format_stamp: "Formatted at 2026-08-08 20:56:23 on dist-test-slave-c12x"
server_key: "2959594663823bb2d31304eb177cbb91"
server_key_iv: "95c2883678c29faa982a16e0cfc758cc"
server_key_version: "encryptionkey@0"
I20260808 20:56:23.174876 19240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:23.187280 19240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:23.189765 19240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:23.189929 19240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:23.190500 19240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:23.190544 19240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.190569 19240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:23.190610 19240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.198019 19240 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:45225
I20260808 20:56:23.198096 19446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:45225 every 8 connection(s)
Aug 08 20:56:23 dist-test-slave-c12x krb5kdc[18856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:23.198457 19240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:23.201802 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.191045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:44993 (local address 127.15.232.190:42047)
0808 20:56:23.191124 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.191127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.191319 (+ 192us) server_negotiation.cc:407] Connection header received
0808 20:56:23.191966 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.191967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.191982 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.192073 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:23.192924 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.193774 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.195401 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.195773 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.195845 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.198850 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:23.198851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:23.198852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:23.198862 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:23.200376 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.200835 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.200836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.200836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.200899 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.201151 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.201152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.201152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.201286 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:23.201354 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.201654 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.201733 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 20:56:23.202000 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.191008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:42047 (local address 127.15.232.130:44993)
0808 20:56:23.191186 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.191228 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.191849 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.192132 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.192136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.192509 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:23.192807 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.192814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.193888 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.193889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.195319 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.195322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.195890 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.195891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.196059 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.196708 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:23.196730 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:23.198727 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:23.200478 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.200485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.200487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.200732 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.200966 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.200967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.200967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.201070 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.201374 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:23.201381 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:23.201526 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.201629 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.201776 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260808 20:56:23.202509 19447 heartbeater.cc:347] Connected to a master server at 127.15.232.190:42047
I20260808 20:56:23.202600 19447 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:23.202778 19447 heartbeater.cc:511] Master 127.15.232.190:42047 requested a full tablet report, sending...
I20260808 20:56:23.203209 18923 ts_manager.cc:194] Registered new tserver with Master: 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225)
I20260808 20:56:23.203765 18923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:44993
I20260808 20:56:23.204411 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 19240
I20260808 20:56:23.204510 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:23.204712 16290 external_mini_cluster.cc:1428] Setting key 0373736c49a81198f9392ec13d5691bb
I20260808 20:56:23.209622 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.203654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:60915 (local address 127.15.232.190:42047)
0808 20:56:23.203695 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.203697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.203841 (+ 144us) server_negotiation.cc:407] Connection header received
0808 20:56:23.203904 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.203905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.203915 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.203963 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:23.204338 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.204850 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.205666 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.205874 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.205946 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.206943 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:23.206945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:23.206945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:23.206954 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:23.208427 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.208829 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.208829 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.208830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.208884 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.209123 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.209124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.209124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.209263 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:23.209317 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.209494 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.209559 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 20:56:23.209774 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.203575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:42047 (local address 127.15.232.130:60915)
0808 20:56:23.203760 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.203772 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.203837 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.203970 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.203970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.204186 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:23.204251 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.204253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.204937 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.204938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.205551 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.205554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.205946 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.205947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.206027 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.206479 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:23.206484 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:23.206832 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:23.208504 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.208505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.208506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.208749 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.208974 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.208975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.208976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.209052 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.209326 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:23.209327 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:23.209384 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.209478 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.209611 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:23.233950 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:42047
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:23.320952 19464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:23.321168 19464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:23.321195 19464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:23.322433 19464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:23.322479 19464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:23.322499 19464 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 20:56:23.322511 19464 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 20:56:23.322554 19464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:23.324285 19464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.190:42047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:23.324535 19464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:23.324807 19464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:23.329890 19471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:23.329859 19470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:23.329898 19476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:23.330271 19464 server_base.cc:1034] running on GCE node
I20260808 20:56:23.330435 19464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:23.330715 19464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:23.331859 19464 hybrid_clock.cc:648] HybridClock initialized: now 1786222583331846 us; error 43 us; skew 500 ppm
Aug 08 20:56:23 dist-test-slave-c12x krb5kdc[18856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:23.334568 19464 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:23.335100 19464 webserver.cc:460] Webserver started at http://127.15.232.131:34255/ using document root <none> and password file <none>
I20260808 20:56:23.335351 19464 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:23.335497 19464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:23.335702 19464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:23.336854 19464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8"
format_stamp: "Formatted at 2026-08-08 20:56:23 on dist-test-slave-c12x"
server_key: "611430a1c71b8350b1717b3a8149c720"
server_key_iv: "32ef0d7d5e4e6532974565db8392b9e5"
server_key_version: "encryptionkey@0"
I20260808 20:56:23.337291 19464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8"
format_stamp: "Formatted at 2026-08-08 20:56:23 on dist-test-slave-c12x"
server_key: "611430a1c71b8350b1717b3a8149c720"
server_key_iv: "32ef0d7d5e4e6532974565db8392b9e5"
server_key_version: "encryptionkey@0"
I20260808 20:56:23.339347 19464 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 20:56:23.340341 19488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.340569 19464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:23.340688 19464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8"
format_stamp: "Formatted at 2026-08-08 20:56:23 on dist-test-slave-c12x"
server_key: "611430a1c71b8350b1717b3a8149c720"
server_key_iv: "32ef0d7d5e4e6532974565db8392b9e5"
server_key_version: "encryptionkey@0"
I20260808 20:56:23.340773 19464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:23.352329 19464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:23.354393 19464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:23.354650 19464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:23.355688 19464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:23.355890 19464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.355934 19464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:23.355954 19464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 20:56:23 dist-test-slave-c12x krb5kdc[18856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222583, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:23.365686 19464 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:43999
I20260808 20:56:23.365763 19656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:43999 every 8 connection(s)
I20260808 20:56:23.366144 19464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:23.367910 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.356040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:55697 (local address 127.15.232.190:42047)
0808 20:56:23.356104 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.356107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.356387 (+ 280us) server_negotiation.cc:407] Connection header received
0808 20:56:23.356926 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.356928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.356940 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.356997 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:23.357613 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.358141 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.359374 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.359772 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.359870 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.364276 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:23.364278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:23.364278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:23.364320 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:23.366331 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.366781 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.366782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.366783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.366853 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.367187 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.367188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.367188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.367324 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:23.367399 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.367725 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.367816 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 20:56:23.368108 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.356044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:42047 (local address 127.15.232.131:55697)
0808 20:56:23.356244 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.356286 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.356817 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.357077 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.357081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.357325 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:23.357518 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.357522 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.358239 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.358241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.359113 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.359115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.359896 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.359897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.360251 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.361482 (+ 1231us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:23.361492 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:23.364169 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:23.366416 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.366420 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.366422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.366690 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.366935 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.366936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.366936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.367107 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.367401 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:23.367409 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:23.367583 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.367691 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.367874 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":116,"threads_started":1}
I20260808 20:56:23.368649 19657 heartbeater.cc:347] Connected to a master server at 127.15.232.190:42047
I20260808 20:56:23.368762 19657 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:23.368984 19657 heartbeater.cc:511] Master 127.15.232.190:42047 requested a full tablet report, sending...
I20260808 20:56:23.369395 18922 ts_manager.cc:194] Registered new tserver with Master: 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999)
I20260808 20:56:23.369838 18922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:55697
I20260808 20:56:23.370893 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 19464
I20260808 20:56:23.371029 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:23.371217 16290 external_mini_cluster.cc:1428] Setting key 4b3e1a8bed31a97a9b5b5110ab63ed0a
I20260808 20:56:23.373332 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:56:23.379858 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.373018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:59279 (local address 127.15.232.190:42047)
0808 20:56:23.373059 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.373061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.373161 (+ 100us) server_negotiation.cc:407] Connection header received
0808 20:56:23.373219 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.373220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.373230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.373278 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:23.373700 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.374189 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.375030 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.375304 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.375348 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.376622 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:23.376624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:23.376624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:23.376632 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:23.378585 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.378933 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.378935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.378935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.379029 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.379230 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.379231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.379232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.379354 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:23.379428 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.379632 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.379728 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 20:56:23.380235 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.372942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:42047 (local address 127.15.232.131:59279)
0808 20:56:23.373085 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.373096 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.373159 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.373321 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.373322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.373558 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:23.373620 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.373622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.374285 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.374286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.374929 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.374932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.375458 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.375459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.375540 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.376087 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:23.376093 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:23.376520 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:23.378668 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.378669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.378669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.378846 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.379092 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.379093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.379093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.379160 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.379420 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:23.379421 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:23.379497 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.379973 (+ 476us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.380105 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
Aug 08 20:56:23 dist-test-slave-c12x krb5kdc[18856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 20:56:23 dist-test-slave-c12x krb5kdc[18856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:23.402631 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.394156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.15.232.190:42047)
0808 20:56:23.394207 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.394209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.394275 (+ 66us) server_negotiation.cc:407] Connection header received
0808 20:56:23.394369 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.394370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.394384 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.394437 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:23.395043 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.395991 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.396779 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.397128 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.397204 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.399659 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:23.399661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:23.399661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:23.399670 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:23.401383 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.401734 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.401735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.401735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.401805 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.401995 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.401996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.401996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.402154 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:23.402269 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.402405 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.402498 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 20:56:23.404526 18923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:23.406347 18923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:23.413988 19685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.409306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:43999 (local address 127.0.0.1:34854)
0808 20:56:23.409563 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.409573 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.409617 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.410109 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.410111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.410113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:23.410160 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.410161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.411368 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.411368 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.412351 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.412353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.413314 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.413315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.413395 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.413410 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.413458 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.413887 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":28,"thread_start_us":75,"threads_started":1}
I20260808 20:56:23.414131 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.409275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48032 (local address 127.15.232.129:38947)
0808 20:56:23.409461 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.409465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.409667 (+ 202us) server_negotiation.cc:407] Connection header received
0808 20:56:23.409716 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.409718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.409733 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.409809 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:23.410075 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.410843 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.412648 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.413522 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.413599 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.413659 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.413954 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.414006 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260808 20:56:23.414726 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.409074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45225 (local address 127.0.0.1:41860)
0808 20:56:23.409447 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.409491 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.409570 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.409892 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.409895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.409897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:23.410009 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.410012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.411361 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.411362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.413433 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.413435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.414477 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.414478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.414540 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.414543 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.414596 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.414641 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":86,"thread_start_us":79,"threads_started":1}
I20260808 20:56:23.414847 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.409108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38947 (local address 127.0.0.1:48032)
0808 20:56:23.409527 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.409540 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.409603 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.409896 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.409897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.409899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:23.409990 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.409992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.410928 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.410929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.412544 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.412547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.413700 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.413701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.413776 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.413780 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.413837 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.413891 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":4,"thread_start_us":98,"threads_started":1}
I20260808 20:56:23.415081 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.409196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.15.232.130:45225)
0808 20:56:23.409355 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.409360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.409666 (+ 306us) server_negotiation.cc:407] Connection header received
0808 20:56:23.409714 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.409716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.409733 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.409805 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:23.410102 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.411263 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.413525 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.414306 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.414381 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.414437 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.414913 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.414964 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260808 20:56:23.415426 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.409590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34854 (local address 127.15.232.131:43999)
0808 20:56:23.409770 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.409775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.409785 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:23.409899 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.409901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.409997 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.410106 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:23.410220 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.411270 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.412448 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.413214 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.414777 (+ 1563us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.415257 (+ 480us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.415285 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.415331 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260808 20:56:23.415740 19128 tablet_service.cc:1467] Processing CreateTablet for tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e (DEFAULT_TABLE table=test-table [id=9a3b352bb77d412bbffe7b1557f9ead3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:23.415846 19338 tablet_service.cc:1467] Processing CreateTablet for tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e (DEFAULT_TABLE table=test-table [id=9a3b352bb77d412bbffe7b1557f9ead3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:23.416177 19128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:23.416263 19338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:23.416316 19564 tablet_service.cc:1467] Processing CreateTablet for tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e (DEFAULT_TABLE table=test-table [id=9a3b352bb77d412bbffe7b1557f9ead3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:23.416632 19564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:23.420463 19698 tablet_bootstrap.cc:492] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: Bootstrap starting.
I20260808 20:56:23.420609 19700 tablet_bootstrap.cc:492] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: Bootstrap starting.
I20260808 20:56:23.421226 19698 tablet_bootstrap.cc:654] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:23.421458 19700 tablet_bootstrap.cc:654] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:23.421608 19698 log.cc:824] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:23.421763 19700 log.cc:824] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:23.422003 19697 tablet_bootstrap.cc:492] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: Bootstrap starting.
I20260808 20:56:23.422540 19700 tablet_bootstrap.cc:492] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: No bootstrap required, opened a new log
I20260808 20:56:23.422643 19700 ts_tablet_manager.cc:1397] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:56:23.422668 19697 tablet_bootstrap.cc:654] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:23.422955 19697 log.cc:824] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:23.423765 19697 tablet_bootstrap.cc:492] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: No bootstrap required, opened a new log
I20260808 20:56:23.423823 19697 ts_tablet_manager.cc:1397] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:23.424746 19700 raft_consensus.cc:348] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } }
I20260808 20:56:23.424963 19700 raft_consensus.cc:374] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:23.425019 19700 raft_consensus.cc:729] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a796dc5f7428797f4a6e23c5bb3f8, State: Initialized, Role: FOLLOWER
I20260808 20:56:23.425170 19700 consensus_queue.cc:260] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } }
I20260808 20:56:23.425450 19700 ts_tablet_manager.cc:1428] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 20:56:23.425549 19657 heartbeater.cc:503] Master 127.15.232.190:42047 was elected leader, sending a full tablet report...
I20260808 20:56:23.425613 19697 raft_consensus.cc:348] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } }
I20260808 20:56:23.425769 19697 raft_consensus.cc:374] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:23.425812 19697 raft_consensus.cc:729] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ff3ef0cf31433a928e08b40d506aac, State: Initialized, Role: FOLLOWER
I20260808 20:56:23.425928 19697 consensus_queue.cc:260] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } }
I20260808 20:56:23.426187 19697 ts_tablet_manager.cc:1428] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 20:56:23.426226 19447 heartbeater.cc:503] Master 127.15.232.190:42047 was elected leader, sending a full tablet report...
I20260808 20:56:23.426656 19698 tablet_bootstrap.cc:492] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: No bootstrap required, opened a new log
I20260808 20:56:23.426733 19698 ts_tablet_manager.cc:1397] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 20:56:23.428447 19698 raft_consensus.cc:348] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } }
I20260808 20:56:23.428575 19698 raft_consensus.cc:374] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:23.428596 19698 raft_consensus.cc:729] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be5243b99494a19b523e2157a1118a8, State: Initialized, Role: FOLLOWER
I20260808 20:56:23.428706 19698 consensus_queue.cc:260] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } }
I20260808 20:56:23.428995 19226 heartbeater.cc:503] Master 127.15.232.190:42047 was elected leader, sending a full tablet report...
I20260808 20:56:23.428970 19698 ts_tablet_manager.cc:1428] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260808 20:56:23.449067 19451 tablet.cc:2369] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:23.452646 19707 raft_consensus.cc:482] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:23.452790 19707 raft_consensus.cc:504] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } }
I20260808 20:56:23.453178 19707 leader_election.cc:288] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225), 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947)
I20260808 20:56:23.457082 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.453577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:53351 (local address 127.15.232.129:38947)
0808 20:56:23.453654 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.453669 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.453678 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:23.453711 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.453712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.453722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.453787 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:23.453982 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.454774 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.456313 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.456696 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.456786 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.456796 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.456968 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.457011 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 20:56:23.457105 19503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.453393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38947 (local address 127.15.232.131:53351)
0808 20:56:23.453470 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.453484 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.453541 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.453810 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.453811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.453814 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:23.453901 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.453903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.454877 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.454878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.456185 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.456189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.456786 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.456787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.456876 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.456880 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.456959 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.457004 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":46}
I20260808 20:56:23.457443 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.453439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:48501 (local address 127.15.232.130:45225)
0808 20:56:23.453494 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.453496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.453787 (+ 291us) server_negotiation.cc:407] Connection header received
0808 20:56:23.453872 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.453873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.453881 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.453975 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:23.454146 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.454820 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.456296 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.456974 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.457053 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.457063 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.457286 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.457336 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 20:56:23.457626 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5d1b4b5474ef29db8c2ebeaf67e6e" candidate_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5243b99494a19b523e2157a1118a8" is_pre_election: true
I20260808 20:56:23.457705 19712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.453396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45225 (local address 127.15.232.131:48501)
0808 20:56:23.453724 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.453735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.453815 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.454004 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.454004 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.454006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:23.454073 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.454074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.454903 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.454903 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.456185 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.456189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.457102 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.457103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.457194 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.457200 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.457540 (+ 340us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.457594 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":162,"thread_start_us":135,"threads_started":1}
I20260808 20:56:23.457827 19139 raft_consensus.cc:2454] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a796dc5f7428797f4a6e23c5bb3f8 in term 0.
I20260808 20:56:23.458019 19378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5d1b4b5474ef29db8c2ebeaf67e6e" candidate_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ff3ef0cf31433a928e08b40d506aac" is_pre_election: true
I20260808 20:56:23.458068 19495 leader_election.cc:302] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a796dc5f7428797f4a6e23c5bb3f8, 7be5243b99494a19b523e2157a1118a8; no voters:
I20260808 20:56:23.458179 19378 raft_consensus.cc:2454] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a796dc5f7428797f4a6e23c5bb3f8 in term 0.
I20260808 20:56:23.458235 19707 raft_consensus.cc:2781] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:23.458299 19707 raft_consensus.cc:482] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:23.458333 19707 raft_consensus.cc:3037] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:23.459084 19707 raft_consensus.cc:504] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } }
I20260808 20:56:23.459239 19707 leader_election.cc:288] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [CANDIDATE]: Term 1 election: Requested vote from peers 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225), 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947)
I20260808 20:56:23.459385 19378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5d1b4b5474ef29db8c2ebeaf67e6e" candidate_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ff3ef0cf31433a928e08b40d506aac"
I20260808 20:56:23.459422 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5d1b4b5474ef29db8c2ebeaf67e6e" candidate_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5243b99494a19b523e2157a1118a8"
I20260808 20:56:23.459503 19378 raft_consensus.cc:3037] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:23.459533 19139 raft_consensus.cc:3037] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:23.460243 19139 raft_consensus.cc:2454] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a796dc5f7428797f4a6e23c5bb3f8 in term 1.
I20260808 20:56:23.460415 19495 leader_election.cc:302] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a796dc5f7428797f4a6e23c5bb3f8, 7be5243b99494a19b523e2157a1118a8; no voters:
I20260808 20:56:23.460520 19707 raft_consensus.cc:2781] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:23.460613 19378 raft_consensus.cc:2454] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a796dc5f7428797f4a6e23c5bb3f8 in term 1.
I20260808 20:56:23.460712 19707 raft_consensus.cc:686] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 LEADER]: Becoming Leader. State: Replica: 5e6a796dc5f7428797f4a6e23c5bb3f8, State: Running, Role: LEADER
I20260808 20:56:23.460815 19707 consensus_queue.cc:237] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } }
I20260808 20:56:23.461730 18922 catalog_manager.cc:5495] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131). New cstate: current_term: 1 leader_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } health_report { overall_health: UNKNOWN } } }
Aug 08 20:56:23 dist-test-slave-c12x krb5kdc[18856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222582, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:23.505153 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.496492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34628 (local address 127.15.232.190:42047)
0808 20:56:23.496567 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.496569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.496751 (+ 182us) server_negotiation.cc:407] Connection header received
0808 20:56:23.496810 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.496810 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.496821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.496879 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:23.497537 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.498134 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.499302 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.499613 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.499673 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.502490 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:23.502491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:23.502492 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:23.502499 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:23.503924 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.504318 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.504319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.504320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.504376 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:23.504570 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:23.504571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:23.504572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:23.504689 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:23.504764 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.504920 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.505029 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 20:56:23.505230 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.496396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:42047 (local address 127.0.0.1:34628)
0808 20:56:23.496644 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.496656 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.496745 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.496907 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.496908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.497380 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:23.497441 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.497445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.498218 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.498220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.499197 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.499201 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.499693 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.499694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.499771 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.500384 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:23.500391 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:23.502348 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:23.504009 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.504019 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.504022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.504228 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.504447 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:23.504447 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:23.504448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:23.504510 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:23.504755 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:23.504764 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:23.504812 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.504895 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.505103 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":80,"thread_start_us":160,"threads_started":1}
I20260808 20:56:23.507655 18922 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34628:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 20:56:23.507937 18922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:23.509498 19338 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccb1297b5944e33838ef21d1f00f808 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b2c53a190b047bb8abc8248ba71c7b2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 20:56:23.509517 19128 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccb1297b5944e33838ef21d1f00f808 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b2c53a190b047bb8abc8248ba71c7b2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 20:56:23.509534 19564 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccb1297b5944e33838ef21d1f00f808 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b2c53a190b047bb8abc8248ba71c7b2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 20:56:23.509701 19564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccb1297b5944e33838ef21d1f00f808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:23.509701 19128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccb1297b5944e33838ef21d1f00f808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:23.509701 19338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccb1297b5944e33838ef21d1f00f808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:23.511493 19698 tablet_bootstrap.cc:492] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: Bootstrap starting.
I20260808 20:56:23.511879 19700 tablet_bootstrap.cc:492] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Bootstrap starting.
I20260808 20:56:23.512233 19698 tablet_bootstrap.cc:654] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:23.512432 19700 tablet_bootstrap.cc:654] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:23.511426 19697 tablet_bootstrap.cc:492] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: Bootstrap starting.
I20260808 20:56:23.513052 19700 tablet_bootstrap.cc:492] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: No bootstrap required, opened a new log
I20260808 20:56:23.513116 19700 ts_tablet_manager.cc:1397] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:23.513238 19697 tablet_bootstrap.cc:654] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:23.513291 19700 raft_consensus.cc:348] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.513370 19700 raft_consensus.cc:374] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:23.513401 19700 raft_consensus.cc:729] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a796dc5f7428797f4a6e23c5bb3f8, State: Initialized, Role: FOLLOWER
I20260808 20:56:23.513451 19700 consensus_queue.cc:260] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.513540 19700 ts_tablet_manager.cc:1428] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:23.513679 19707 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.513762 19707 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:23.514122 19698 tablet_bootstrap.cc:492] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: No bootstrap required, opened a new log
I20260808 20:56:23.514164 19698 ts_tablet_manager.cc:1397] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 20:56:23.514333 19698 raft_consensus.cc:348] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.514395 19698 raft_consensus.cc:374] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:23.514436 19698 raft_consensus.cc:729] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be5243b99494a19b523e2157a1118a8, State: Initialized, Role: FOLLOWER
I20260808 20:56:23.514492 19698 consensus_queue.cc:260] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.514599 19698 ts_tablet_manager.cc:1428] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.514681 19711 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.514786 19711 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:23.514900 19697 tablet_bootstrap.cc:492] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: No bootstrap required, opened a new log
I20260808 20:56:23.514955 19697 ts_tablet_manager.cc:1397] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 20:56:23.515112 19697 raft_consensus.cc:348] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.515190 19697 raft_consensus.cc:374] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:23.515215 19697 raft_consensus.cc:729] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ff3ef0cf31433a928e08b40d506aac, State: Initialized, Role: FOLLOWER
I20260808 20:56:23.515268 19697 consensus_queue.cc:260] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.515367 19697 ts_tablet_manager.cc:1428] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:23.515460 19710 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.515575 19710 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 20:56:23.528524 19227 tablet.cc:2369] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:23.544834 19711 raft_consensus.cc:482] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:23.544947 19711 raft_consensus.cc:504] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.545329 19711 leader_election.cc:288] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225), 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999)
I20260808 20:56:23.548504 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.545594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:38499 (local address 127.15.232.130:45225)
0808 20:56:23.545680 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.545682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.545779 (+ 97us) server_negotiation.cc:407] Connection header received
0808 20:56:23.545856 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.545858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.545868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.545935 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:23.546100 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.546636 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.547786 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.548149 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.548237 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.548245 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.548380 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.548428 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 20:56:23.548512 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.545813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:41781 (local address 127.15.232.131:43999)
0808 20:56:23.545859 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.545861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.545878 (+ 17us) server_negotiation.cc:407] Connection header received
0808 20:56:23.545967 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.545968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.545977 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.546041 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:23.546210 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.546691 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.547755 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.548150 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.548231 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.548241 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.548405 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.548445 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 20:56:23.548528 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.545524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45225 (local address 127.15.232.129:38499)
0808 20:56:23.545674 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.545689 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.545778 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.545952 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.545953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.545956 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:23.546024 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.546026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.546723 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.546724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.547676 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.547679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.548236 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.548237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.548305 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.548309 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.548352 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.548433 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":56,"thread_start_us":76,"threads_started":1}
I20260808 20:56:23.548615 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.545729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:43999 (local address 127.15.232.129:41781)
0808 20:56:23.545827 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.545838 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.545885 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.546051 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.546052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.546054 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:23.546142 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.546144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.546781 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.546782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.547671 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.547674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.548242 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.548243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.548318 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.548321 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.548390 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.548465 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":37,"thread_start_us":45,"threads_started":1}
I20260808 20:56:23.548804 19378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccb1297b5944e33838ef21d1f00f808" candidate_uuid: "7be5243b99494a19b523e2157a1118a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ff3ef0cf31433a928e08b40d506aac" is_pre_election: true
I20260808 20:56:23.548898 19378 raft_consensus.cc:2454] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be5243b99494a19b523e2157a1118a8 in term 0.
I20260808 20:56:23.548939 19595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccb1297b5944e33838ef21d1f00f808" candidate_uuid: "7be5243b99494a19b523e2157a1118a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" is_pre_election: true
I20260808 20:56:23.549074 19595 raft_consensus.cc:2454] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be5243b99494a19b523e2157a1118a8 in term 0.
I20260808 20:56:23.549150 19054 leader_election.cc:302] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ff3ef0cf31433a928e08b40d506aac, 7be5243b99494a19b523e2157a1118a8; no voters:
I20260808 20:56:23.549301 19711 raft_consensus.cc:2781] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:23.549350 19711 raft_consensus.cc:482] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:23.549386 19711 raft_consensus.cc:3037] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:23.550158 19711 raft_consensus.cc:504] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.550307 19711 leader_election.cc:288] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [CANDIDATE]: Term 1 election: Requested vote from peers 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225), 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999)
I20260808 20:56:23.550489 19378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccb1297b5944e33838ef21d1f00f808" candidate_uuid: "7be5243b99494a19b523e2157a1118a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ff3ef0cf31433a928e08b40d506aac"
I20260808 20:56:23.550513 19595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccb1297b5944e33838ef21d1f00f808" candidate_uuid: "7be5243b99494a19b523e2157a1118a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8"
I20260808 20:56:23.550587 19595 raft_consensus.cc:3037] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:23.550590 19378 raft_consensus.cc:3037] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:23.551266 19378 raft_consensus.cc:2454] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be5243b99494a19b523e2157a1118a8 in term 1.
I20260808 20:56:23.551271 19595 raft_consensus.cc:2454] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be5243b99494a19b523e2157a1118a8 in term 1.
I20260808 20:56:23.551503 19054 leader_election.cc:302] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a796dc5f7428797f4a6e23c5bb3f8, 7be5243b99494a19b523e2157a1118a8; no voters:
I20260808 20:56:23.551646 19711 raft_consensus.cc:2781] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:23.551985 19711 raft_consensus.cc:686] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [term 1 LEADER]: Becoming Leader. State: Replica: 7be5243b99494a19b523e2157a1118a8, State: Running, Role: LEADER
I20260808 20:56:23.552115 19711 consensus_queue.cc:237] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.552632 19739 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: TxnStatusTablet state changed. Reason: New leader 7be5243b99494a19b523e2157a1118a8. Latest consensus state: current_term: 1 leader_uuid: "7be5243b99494a19b523e2157a1118a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.552821 19739 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: This TxnStatusTablet replica's current role is: LEADER
I20260808 20:56:23.553081 19742 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 20:56:23.553190 19742 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 20:56:23.553048 18923 catalog_manager.cc:5495] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7be5243b99494a19b523e2157a1118a8 (127.15.232.129). New cstate: current_term: 1 leader_uuid: "7be5243b99494a19b523e2157a1118a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } health_report { overall_health: UNKNOWN } } }
I20260808 20:56:23.565078 19728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.561786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38947 (local address 127.0.0.1:48044)
0808 20:56:23.561900 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.561913 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.561966 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.562306 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.562307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.562308 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:23.562358 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.562360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.563310 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.563311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.564004 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.564007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.564486 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.564487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.564544 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.564562 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.564887 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.564899 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.564947 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.565012 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 20:56:23.565069 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.562057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.15.232.129:38947)
0808 20:56:23.562130 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.562132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.562142 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:23.562177 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.562178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.562199 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.562275 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:23.562432 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.563216 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.564081 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.564396 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.564475 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.564655 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.564788 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.564886 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.564961 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.564984 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 20:56:23.567200 19595 raft_consensus.cc:1261] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Refusing update from remote peer 7be5243b99494a19b523e2157a1118a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:23.567220 19378 raft_consensus.cc:1261] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Refusing update from remote peer 7be5243b99494a19b523e2157a1118a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:23.567622 19711 consensus_queue.cc:1035] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:23.567777 19739 consensus_queue.cc:1035] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:23.569380 19707 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: TxnStatusTablet state changed. Reason: New leader 7be5243b99494a19b523e2157a1118a8. Latest consensus state: current_term: 1 leader_uuid: "7be5243b99494a19b523e2157a1118a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.569481 19707 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:23.570308 19711 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7be5243b99494a19b523e2157a1118a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.570423 19711 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: This TxnStatusTablet replica's current role is: LEADER
I20260808 20:56:23.571015 19746 mvcc.cc:205] Tried to move back new op lower bound from 7316367702288408576 to 7316367702230151168. Current Snapshot: MvccSnapshot[applied={T|T < 7316367702288408576}]
I20260808 20:56:23.571216 19747 mvcc.cc:205] Tried to move back new op lower bound from 7316367702288408576 to 7316367702230151168. Current Snapshot: MvccSnapshot[applied={T|T < 7316367702288408576}]
I20260808 20:56:23.571233 19710 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: TxnStatusTablet state changed. Reason: New leader 7be5243b99494a19b523e2157a1118a8. Latest consensus state: current_term: 1 leader_uuid: "7be5243b99494a19b523e2157a1118a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.571352 19710 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:23.571362 19707 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7be5243b99494a19b523e2157a1118a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.571484 19707 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 5e6a796dc5f7428797f4a6e23c5bb3f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:23.572549 19739 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7be5243b99494a19b523e2157a1118a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.572638 19739 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 7be5243b99494a19b523e2157a1118a8: This TxnStatusTablet replica's current role is: LEADER
I20260808 20:56:23.572973 19710 tablet_replica.cc:439] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7be5243b99494a19b523e2157a1118a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } } }
I20260808 20:56:23.573063 19710 tablet_replica.cc:442] T 1ccb1297b5944e33838ef21d1f00f808 P 12ff3ef0cf31433a928e08b40d506aac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:23.573935 19751 mvcc.cc:205] Tried to move back new op lower bound from 7316367702288408576 to 7316367702230151168. Current Snapshot: MvccSnapshot[applied={T|T < 7316367702288408576}]
I20260808 20:56:23.577026 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.574256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.15.232.131:43999)
0808 20:56:23.574294 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.574297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.574306 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:23.574375 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.574376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.574392 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.574445 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:23.574585 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.575074 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.576243 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.576444 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.576511 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.576713 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.576817 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.576888 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.576953 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.576973 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 20:56:23.582810 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.579884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:53953 (local address 127.15.232.129:38947)
0808 20:56:23.579926 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.579928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.579953 (+ 25us) server_negotiation.cc:407] Connection header received
0808 20:56:23.579976 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.579977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.579985 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.580034 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:23.580170 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.580639 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.581833 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.582117 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.582237 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.582517 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.582568 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.582625 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.582704 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.582735 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 20:56:23.582832 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.579776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38947 (local address 127.15.232.131:53953)
0808 20:56:23.579856 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.579867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.579904 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.580045 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.580045 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.580046 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:23.580109 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.580110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.580712 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.580713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.581648 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.581651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.582248 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.582249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.582332 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.582350 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.582642 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.582645 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.582701 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.582751 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 20:56:23.585417 19378 raft_consensus.cc:1261] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Refusing update from remote peer 5e6a796dc5f7428797f4a6e23c5bb3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:23.585438 19139 raft_consensus.cc:1261] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Refusing update from remote peer 5e6a796dc5f7428797f4a6e23c5bb3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:23.585675 19716 consensus_queue.cc:1035] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:23.585836 19707 consensus_queue.cc:1035] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:23.596243 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.593134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:43999 (local address 127.15.232.129:33865)
0808 20:56:23.593341 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.593354 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.593413 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.593603 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.593605 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.593606 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:23.593693 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.593695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.594606 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.594607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.595218 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.595220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.595729 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.595730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.595809 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.595842 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.596057 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.596060 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.596107 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.596151 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260808 20:56:23.596233 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.593385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:33865 (local address 127.15.232.131:43999)
0808 20:56:23.593421 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.593423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.593434 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:23.593517 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.593519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.593529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.593594 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:23.593766 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.594503 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.595306 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.595618 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.595688 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.595953 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.595987 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.596055 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.596108 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.596130 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 20:56:23.605405 18922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:23.605541 18922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:23.606398 18922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7: Sending DeleteTablet for 3 replicas of tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e
I20260808 20:56:23.606792 19128 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:23 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:48032
I20260808 20:56:23.606950 19338 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:23 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:41860
I20260808 20:56:23.607208 19774 tablet_replica.cc:330] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: stopping tablet replica
I20260808 20:56:23.607229 19564 tablet_service.cc:1514] Processing DeleteTablet for tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:23 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34854
I20260808 20:56:23.607424 19774 raft_consensus.cc:2229] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:23.607573 19774 raft_consensus.cc:2258] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:23.607754 18922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:23.607918 18922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:23.608363 19775 tablet_replica.cc:330] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: stopping tablet replica
I20260808 20:56:23.608491 19775 raft_consensus.cc:2229] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:23.608544 19775 raft_consensus.cc:2258] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:23.608695 19776 tablet_replica.cc:330] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: stopping tablet replica
I20260808 20:56:23.608779 19776 raft_consensus.cc:2229] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:23.608868 19776 raft_consensus.cc:2258] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:23.609441 19774 ts_tablet_manager.cc:1905] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:23.609589 19128 tablet_service.cc:1467] Processing CreateTablet for tablet 319ee39035914905a644c57c44715053 (DEFAULT_TABLE table=test-table [id=dbdc1ca0500e4550817e12de1399af89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:23.609735 19128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 319ee39035914905a644c57c44715053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:23.611157 19698 tablet_bootstrap.cc:492] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: Bootstrap starting.
I20260808 20:56:23.611626 19698 tablet_bootstrap.cc:654] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:23.612210 19774 ts_tablet_manager.cc:1918] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:23.612344 19774 log.cc:1196] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/f2f5d1b4b5474ef29db8c2ebeaf67e6e
I20260808 20:56:23.612409 19338 tablet_service.cc:1467] Processing CreateTablet for tablet 319ee39035914905a644c57c44715053 (DEFAULT_TABLE table=test-table [id=dbdc1ca0500e4550817e12de1399af89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:23.612550 19338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 319ee39035914905a644c57c44715053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:23.612856 19774 ts_tablet_manager.cc:1939] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 7be5243b99494a19b523e2157a1118a8: Deleting consensus metadata
I20260808 20:56:23.613349 18899 catalog_manager.cc:4841] TS 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947): tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e (table test-table [id=9a3b352bb77d412bbffe7b1557f9ead3]) successfully deleted
I20260808 20:56:23.613934 19697 tablet_bootstrap.cc:492] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: Bootstrap starting.
I20260808 20:56:23.614013 19564 tablet_service.cc:1467] Processing CreateTablet for tablet 319ee39035914905a644c57c44715053 (DEFAULT_TABLE table=test-table [id=dbdc1ca0500e4550817e12de1399af89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:23.614128 19564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 319ee39035914905a644c57c44715053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:23.614346 19697 tablet_bootstrap.cc:654] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:23.615993 19700 tablet_bootstrap.cc:492] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Bootstrap starting.
I20260808 20:56:23.616421 19700 tablet_bootstrap.cc:654] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:23.617497 19775 ts_tablet_manager.cc:1905] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:23.617805 19700 tablet_bootstrap.cc:492] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: No bootstrap required, opened a new log
I20260808 20:56:23.617887 19700 ts_tablet_manager.cc:1397] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:23.618242 19700 raft_consensus.cc:348] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.618423 19700 raft_consensus.cc:374] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:23.618641 19700 raft_consensus.cc:729] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a796dc5f7428797f4a6e23c5bb3f8, State: Initialized, Role: FOLLOWER
I20260808 20:56:23.618827 19700 consensus_queue.cc:260] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.619226 19775 ts_tablet_manager.cc:1918] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:23.619307 19775 log.cc:1196] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/f2f5d1b4b5474ef29db8c2ebeaf67e6e
I20260808 20:56:23.619655 19776 ts_tablet_manager.cc:1905] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:23.620543 19700 ts_tablet_manager.cc:1428] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 20:56:23.623237 19776 ts_tablet_manager.cc:1918] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:23.623320 19776 log.cc:1196] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/f2f5d1b4b5474ef29db8c2ebeaf67e6e
I20260808 20:56:23.623745 19775 ts_tablet_manager.cc:1939] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 12ff3ef0cf31433a928e08b40d506aac: Deleting consensus metadata
I20260808 20:56:23.624230 18896 catalog_manager.cc:4841] TS 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225): tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e (table test-table [id=9a3b352bb77d412bbffe7b1557f9ead3]) successfully deleted
I20260808 20:56:23.624694 19776 ts_tablet_manager.cc:1939] T f2f5d1b4b5474ef29db8c2ebeaf67e6e P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting consensus metadata
I20260808 20:56:23.625064 18896 catalog_manager.cc:4841] TS 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999): tablet f2f5d1b4b5474ef29db8c2ebeaf67e6e (table test-table [id=9a3b352bb77d412bbffe7b1557f9ead3]) successfully deleted
I20260808 20:56:23.625312 19698 tablet_bootstrap.cc:492] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: No bootstrap required, opened a new log
I20260808 20:56:23.625376 19698 ts_tablet_manager.cc:1397] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20260808 20:56:23.625536 19698 raft_consensus.cc:348] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.625604 19698 raft_consensus.cc:374] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:23.625622 19698 raft_consensus.cc:729] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be5243b99494a19b523e2157a1118a8, State: Initialized, Role: FOLLOWER
I20260808 20:56:23.625664 19698 consensus_queue.cc:260] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.625752 19698 ts_tablet_manager.cc:1428] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:23.632071 19697 tablet_bootstrap.cc:492] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: No bootstrap required, opened a new log
I20260808 20:56:23.632135 19697 ts_tablet_manager.cc:1397] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20260808 20:56:23.632325 19697 raft_consensus.cc:348] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.632417 19697 raft_consensus.cc:374] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:23.632453 19697 raft_consensus.cc:729] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ff3ef0cf31433a928e08b40d506aac, State: Initialized, Role: FOLLOWER
I20260808 20:56:23.632503 19697 consensus_queue.cc:260] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.632614 19697 ts_tablet_manager.cc:1428] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:23.775782 19710 raft_consensus.cc:482] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:23.775910 19710 raft_consensus.cc:504] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.776321 19710 leader_election.cc:288] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947), 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999)
I20260808 20:56:23.779525 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.776623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:44769 (local address 127.15.232.131:43999)
0808 20:56:23.776681 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.776683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.776755 (+ 72us) server_negotiation.cc:407] Connection header received
0808 20:56:23.776834 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.776835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.776849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.776899 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:23.777076 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.777567 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.778657 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.779140 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.779196 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.779204 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.779358 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.779397 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 20:56:23.779531 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.776532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:43999 (local address 127.15.232.130:44769)
0808 20:56:23.776677 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.776687 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.776771 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.776924 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.776925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.776927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:23.777008 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.777010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.777646 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.777647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.778566 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.778569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.779204 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.779205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.779279 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.779285 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.779340 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.779383 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":1,"thread_start_us":81,"threads_started":1}
I20260808 20:56:23.779671 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.776534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38947 (local address 127.15.232.130:46479)
0808 20:56:23.776695 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.776706 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.776775 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.776960 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.776961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.776963 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:23.777033 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.777049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.777673 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.777674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.778577 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.778580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.779350 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.779351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.779470 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.779476 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.779535 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.779581 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":117,"thread_start_us":72,"threads_started":1}
I20260808 20:56:23.779680 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.776611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:46479 (local address 127.15.232.129:38947)
0808 20:56:23.776671 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.776673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.776776 (+ 103us) server_negotiation.cc:407] Connection header received
0808 20:56:23.776844 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.776845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.776857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.776927 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:23.777117 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.777569 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.778694 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.779242 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.779318 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.779328 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.779543 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.779595 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 20:56:23.779769 19595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319ee39035914905a644c57c44715053" candidate_uuid: "12ff3ef0cf31433a928e08b40d506aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" is_pre_election: true
I20260808 20:56:23.779865 19595 raft_consensus.cc:2454] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ff3ef0cf31433a928e08b40d506aac in term 0.
I20260808 20:56:23.779907 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319ee39035914905a644c57c44715053" candidate_uuid: "12ff3ef0cf31433a928e08b40d506aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5243b99494a19b523e2157a1118a8" is_pre_election: true
I20260808 20:56:23.780015 19139 raft_consensus.cc:2454] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ff3ef0cf31433a928e08b40d506aac in term 0.
I20260808 20:56:23.780041 19265 leader_election.cc:302] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ff3ef0cf31433a928e08b40d506aac, 5e6a796dc5f7428797f4a6e23c5bb3f8; no voters:
I20260808 20:56:23.780170 19710 raft_consensus.cc:2781] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:23.780197 19710 raft_consensus.cc:482] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:23.780216 19710 raft_consensus.cc:3037] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:23.780900 19710 raft_consensus.cc:504] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.781044 19710 leader_election.cc:288] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [CANDIDATE]: Term 1 election: Requested vote from peers 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947), 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999)
I20260808 20:56:23.781190 19595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319ee39035914905a644c57c44715053" candidate_uuid: "12ff3ef0cf31433a928e08b40d506aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8"
I20260808 20:56:23.781281 19595 raft_consensus.cc:3037] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:23.781239 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319ee39035914905a644c57c44715053" candidate_uuid: "12ff3ef0cf31433a928e08b40d506aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5243b99494a19b523e2157a1118a8"
I20260808 20:56:23.781330 19139 raft_consensus.cc:3037] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:23.781911 19595 raft_consensus.cc:2454] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ff3ef0cf31433a928e08b40d506aac in term 1.
I20260808 20:56:23.782110 19139 raft_consensus.cc:2454] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ff3ef0cf31433a928e08b40d506aac in term 1.
I20260808 20:56:23.782124 19265 leader_election.cc:302] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ff3ef0cf31433a928e08b40d506aac, 5e6a796dc5f7428797f4a6e23c5bb3f8; no voters:
I20260808 20:56:23.782254 19710 raft_consensus.cc:2781] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:23.782372 19710 raft_consensus.cc:686] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 1 LEADER]: Becoming Leader. State: Replica: 12ff3ef0cf31433a928e08b40d506aac, State: Running, Role: LEADER
I20260808 20:56:23.782452 19710 consensus_queue.cc:237] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:23.783198 18923 catalog_manager.cc:5495] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac reported cstate change: term changed from 0 to 1, leader changed from <none> to 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130). New cstate: current_term: 1 leader_uuid: "12ff3ef0cf31433a928e08b40d506aac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } health_report { overall_health: UNKNOWN } } }
I20260808 20:56:23.811976 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.809149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.15.232.130:45225)
0808 20:56:23.809193 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.809195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.809203 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:56:23.809305 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.809306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.809332 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.809460 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:23.809588 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.810078 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.811102 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.811316 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.811397 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.811576 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.811711 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.811792 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.811847 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.811872 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 20:56:23.817801 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.814797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:54383 (local address 127.15.232.129:38947)
0808 20:56:23.814907 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.814909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.814998 (+ 89us) server_negotiation.cc:407] Connection header received
0808 20:56:23.815075 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.815076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.815087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.815147 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:23.815327 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.815889 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.816881 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.817086 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.817160 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.817353 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.817395 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.817450 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.817680 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.817710 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 20:56:23.817842 19792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.814687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38947 (local address 127.15.232.130:54383)
0808 20:56:23.814912 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.814924 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.815015 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.815150 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.815151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.815152 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:23.815234 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.815236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.815974 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.815976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.816769 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.816772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.817171 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.817173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.817251 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.817268 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.817467 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.817470 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.817659 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.817712 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
I20260808 20:56:23.820806 19139 raft_consensus.cc:1261] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Refusing update from remote peer 12ff3ef0cf31433a928e08b40d506aac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:23.820839 19595 raft_consensus.cc:1261] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Refusing update from remote peer 12ff3ef0cf31433a928e08b40d506aac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:23.821156 19787 consensus_queue.cc:1035] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:23.821300 19710 consensus_queue.cc:1035] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:23.829253 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.826790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:58577 (local address 127.15.232.130:45225)
0808 20:56:23.826832 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:23.826834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:23.826843 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:23.826907 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:23.826908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:23.826918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:23.826973 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:23.827147 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.827742 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.828568 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.828765 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.828841 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.828981 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.829018 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.829084 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:23.829156 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 20:56:23.829177 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 20:56:23.829267 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:23.826749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45225 (local address 127.15.232.129:58577)
0808 20:56:23.826796 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:23.826807 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:23.826852 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:23.826975 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:23.826976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:23.826977 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:23.827035 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.827037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.827810 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.827811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.828433 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:23.828435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:23.828841 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:23.828842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:23.828899 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:23.828906 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:23.829087 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:23.829089 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 20:56:23.829151 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 20:56:23.829200 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 20:56:23.829634 19751 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 20:56:25.354971 18923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:25.355083 18923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:25.356094 18923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7: Sending DeleteTablet for 3 replicas of tablet 319ee39035914905a644c57c44715053
I20260808 20:56:25.356382 19564 tablet_service.cc:1514] Processing DeleteTablet for tablet 319ee39035914905a644c57c44715053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:25 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34854
I20260808 20:56:25.356410 19338 tablet_service.cc:1514] Processing DeleteTablet for tablet 319ee39035914905a644c57c44715053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:25 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:41860
I20260808 20:56:25.356609 19832 tablet_replica.cc:330] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: stopping tablet replica
I20260808 20:56:25.356600 19128 tablet_service.cc:1514] Processing DeleteTablet for tablet 319ee39035914905a644c57c44715053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:25 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:48032
I20260808 20:56:25.356678 19832 raft_consensus.cc:2229] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:25.356732 19832 raft_consensus.cc:2258] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:25.356989 19832 ts_tablet_manager.cc:1905] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:25.359182 19832 ts_tablet_manager.cc:1918] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 20:56:25.359264 19832 log.cc:1196] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/319ee39035914905a644c57c44715053
I20260808 20:56:25.359546 19832 ts_tablet_manager.cc:1939] T 319ee39035914905a644c57c44715053 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting consensus metadata
I20260808 20:56:25.360015 18896 catalog_manager.cc:4841] TS 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999): tablet 319ee39035914905a644c57c44715053 (table test-table [id=dbdc1ca0500e4550817e12de1399af89]) successfully deleted
I20260808 20:56:25.360364 19833 tablet_replica.cc:330] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: stopping tablet replica
I20260808 20:56:25.360424 19833 raft_consensus.cc:2229] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:25.360553 19833 raft_consensus.cc:2258] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:25.360831 19833 ts_tablet_manager.cc:1905] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:25.362095 19838 tablet_replica.cc:330] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: stopping tablet replica
I20260808 20:56:25.362183 19838 raft_consensus.cc:2229] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:25.362252 19838 raft_consensus.cc:2258] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:25.362213 19833 ts_tablet_manager.cc:1918] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 20:56:25.362341 19833 log.cc:1196] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/319ee39035914905a644c57c44715053
I20260808 20:56:25.362587 19838 ts_tablet_manager.cc:1905] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:25.362666 19833 ts_tablet_manager.cc:1939] T 319ee39035914905a644c57c44715053 P 12ff3ef0cf31433a928e08b40d506aac: Deleting consensus metadata
I20260808 20:56:25.363004 18896 catalog_manager.cc:4841] TS 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225): tablet 319ee39035914905a644c57c44715053 (table test-table [id=dbdc1ca0500e4550817e12de1399af89]) successfully deleted
I20260808 20:56:25.364552 19838 ts_tablet_manager.cc:1918] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 20:56:25.364639 19838 log.cc:1196] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/319ee39035914905a644c57c44715053
I20260808 20:56:25.364845 19838 ts_tablet_manager.cc:1939] T 319ee39035914905a644c57c44715053 P 7be5243b99494a19b523e2157a1118a8: Deleting consensus metadata
I20260808 20:56:25.365186 18899 catalog_manager.cc:4841] TS 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947): tablet 319ee39035914905a644c57c44715053 (table test-table [id=dbdc1ca0500e4550817e12de1399af89]) successfully deleted
Aug 08 20:56:25 dist-test-slave-c12x krb5kdc[18856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222585, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 20:56:25 dist-test-slave-c12x krb5kdc[18856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222585, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:25.386567 19850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:25.375550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.15.232.190:42047)
0808 20:56:25.375701 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:25.375703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:25.375726 (+ 23us) server_negotiation.cc:407] Connection header received
0808 20:56:25.375800 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:25.375800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:25.375811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:25.375883 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:25.376472 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:25.376985 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:25.377757 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:25.377954 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:25.378073 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:25.383905 (+ 5832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:25.383907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:25.383907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:25.383916 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:25.385297 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:25.385680 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:25.385681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:25.385681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:25.385776 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:25.385992 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:25.385993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:25.385993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:25.386116 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:25.386194 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:25.386338 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 20:56:25.386455 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260808 20:56:25.388268 18923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:25.388440 18923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:25.389830 19128 tablet_service.cc:1467] Processing CreateTablet for tablet 4c3751775cb14f5bae080558e4497b21 (DEFAULT_TABLE table=test-table [id=276b463ed9ef4623afbfa91beba1a77f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:25.389874 19564 tablet_service.cc:1467] Processing CreateTablet for tablet 4c3751775cb14f5bae080558e4497b21 (DEFAULT_TABLE table=test-table [id=276b463ed9ef4623afbfa91beba1a77f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:25.389881 19338 tablet_service.cc:1467] Processing CreateTablet for tablet 4c3751775cb14f5bae080558e4497b21 (DEFAULT_TABLE table=test-table [id=276b463ed9ef4623afbfa91beba1a77f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:25.389986 19128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c3751775cb14f5bae080558e4497b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:25.389986 19564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c3751775cb14f5bae080558e4497b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:25.390049 19338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c3751775cb14f5bae080558e4497b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:25.391403 19851 tablet_bootstrap.cc:492] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: Bootstrap starting.
I20260808 20:56:25.391583 19853 tablet_bootstrap.cc:492] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Bootstrap starting.
I20260808 20:56:25.391659 19852 tablet_bootstrap.cc:492] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: Bootstrap starting.
I20260808 20:56:25.392037 19852 tablet_bootstrap.cc:654] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:25.392037 19853 tablet_bootstrap.cc:654] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:25.392071 19851 tablet_bootstrap.cc:654] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:25.392661 19852 tablet_bootstrap.cc:492] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: No bootstrap required, opened a new log
I20260808 20:56:25.392675 19853 tablet_bootstrap.cc:492] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: No bootstrap required, opened a new log
I20260808 20:56:25.392706 19852 ts_tablet_manager.cc:1397] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:25.392733 19853 ts_tablet_manager.cc:1397] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:25.392858 19853 raft_consensus.cc:348] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.392858 19852 raft_consensus.cc:348] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.392931 19852 raft_consensus.cc:374] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:25.392935 19853 raft_consensus.cc:374] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:25.392946 19852 raft_consensus.cc:729] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be5243b99494a19b523e2157a1118a8, State: Initialized, Role: FOLLOWER
I20260808 20:56:25.392962 19853 raft_consensus.cc:729] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a796dc5f7428797f4a6e23c5bb3f8, State: Initialized, Role: FOLLOWER
I20260808 20:56:25.393006 19852 consensus_queue.cc:260] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.393105 19852 ts_tablet_manager.cc:1428] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:25.393093 19853 consensus_queue.cc:260] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.393164 19860 raft_consensus.cc:482] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:25.393217 19860 raft_consensus.cc:504] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.393262 19853 ts_tablet_manager.cc:1428] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:25.393350 19851 tablet_bootstrap.cc:492] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: No bootstrap required, opened a new log
I20260808 20:56:25.393405 19851 ts_tablet_manager.cc:1397] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:25.393464 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c3751775cb14f5bae080558e4497b21" candidate_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5243b99494a19b523e2157a1118a8" is_pre_election: true
I20260808 20:56:25.393523 19139 raft_consensus.cc:2454] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a796dc5f7428797f4a6e23c5bb3f8 in term 0.
I20260808 20:56:25.393510 19851 raft_consensus.cc:348] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.393589 19851 raft_consensus.cc:374] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:25.393620 19851 raft_consensus.cc:729] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ff3ef0cf31433a928e08b40d506aac, State: Initialized, Role: FOLLOWER
I20260808 20:56:25.393652 19378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c3751775cb14f5bae080558e4497b21" candidate_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ff3ef0cf31433a928e08b40d506aac" is_pre_election: true
I20260808 20:56:25.393658 19851 consensus_queue.cc:260] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.393766 19378 raft_consensus.cc:2454] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a796dc5f7428797f4a6e23c5bb3f8 in term 0.
I20260808 20:56:25.393918 19851 ts_tablet_manager.cc:1428] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:25.394110 19495 leader_election.cc:302] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a796dc5f7428797f4a6e23c5bb3f8, 7be5243b99494a19b523e2157a1118a8; no voters:
I20260808 20:56:25.394330 19860 leader_election.cc:288] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947), 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225)
I20260808 20:56:25.394413 19860 raft_consensus.cc:2781] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:25.394445 19860 raft_consensus.cc:482] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:25.394475 19860 raft_consensus.cc:3037] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:25.395201 19860 raft_consensus.cc:504] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.395346 19860 leader_election.cc:288] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [CANDIDATE]: Term 1 election: Requested vote from peers 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947), 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225)
I20260808 20:56:25.395533 19378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c3751775cb14f5bae080558e4497b21" candidate_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ff3ef0cf31433a928e08b40d506aac"
I20260808 20:56:25.395525 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c3751775cb14f5bae080558e4497b21" candidate_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5243b99494a19b523e2157a1118a8"
I20260808 20:56:25.395599 19378 raft_consensus.cc:3037] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:25.395599 19139 raft_consensus.cc:3037] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:25.396867 19139 raft_consensus.cc:2454] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a796dc5f7428797f4a6e23c5bb3f8 in term 1.
I20260808 20:56:25.396335 19378 raft_consensus.cc:2454] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a796dc5f7428797f4a6e23c5bb3f8 in term 1.
I20260808 20:56:25.397322 19495 leader_election.cc:302] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a796dc5f7428797f4a6e23c5bb3f8, 7be5243b99494a19b523e2157a1118a8; no voters:
I20260808 20:56:25.397511 19861 raft_consensus.cc:2781] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:25.397595 19861 raft_consensus.cc:686] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 LEADER]: Becoming Leader. State: Replica: 5e6a796dc5f7428797f4a6e23c5bb3f8, State: Running, Role: LEADER
I20260808 20:56:25.397645 19861 consensus_queue.cc:237] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.398244 18922 catalog_manager.cc:5495] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131). New cstate: current_term: 1 leader_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } health_report { overall_health: HEALTHY } } }
I20260808 20:56:25.407927 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:25.405141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.15.232.131:43999)
0808 20:56:25.405289 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:25.405291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:25.405296 (+ 5us) server_negotiation.cc:407] Connection header received
0808 20:56:25.405314 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:25.405314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:25.405324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:25.405414 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:25.405695 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:25.406265 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:25.407207 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:25.407416 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:25.407545 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:25.407704 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:25.407745 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:25.407796 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:25.407840 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 20:56:25.407862 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":102,"threads_started":1}
I20260808 20:56:25.410316 19378 raft_consensus.cc:1261] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Refusing update from remote peer 5e6a796dc5f7428797f4a6e23c5bb3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:25.410316 19139 raft_consensus.cc:1261] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Refusing update from remote peer 5e6a796dc5f7428797f4a6e23c5bb3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:25.410532 19861 consensus_queue.cc:1035] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:25.410624 19860 consensus_queue.cc:1035] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:25.421155 18922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:25.421257 18922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:25.421856 18922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7: Sending DeleteTablet for 3 replicas of tablet 4c3751775cb14f5bae080558e4497b21
I20260808 20:56:25.422113 19338 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c3751775cb14f5bae080558e4497b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:25 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:41860
I20260808 20:56:25.422156 19128 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c3751775cb14f5bae080558e4497b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:25 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:48032
I20260808 20:56:25.422154 19564 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c3751775cb14f5bae080558e4497b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:25 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34854
I20260808 20:56:25.422246 19833 tablet_replica.cc:330] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: stopping tablet replica
I20260808 20:56:25.422246 19838 tablet_replica.cc:330] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: stopping tablet replica
I20260808 20:56:25.422261 19832 tablet_replica.cc:330] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: stopping tablet replica
I20260808 20:56:25.422300 19833 raft_consensus.cc:2229] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:25.422310 19832 raft_consensus.cc:2229] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:25.422307 19838 raft_consensus.cc:2229] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:25.422344 19833 raft_consensus.cc:2258] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:25.422364 19838 raft_consensus.cc:2258] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:25.422389 19832 raft_consensus.cc:2258] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:25.422708 19838 ts_tablet_manager.cc:1905] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:25.422726 19833 ts_tablet_manager.cc:1905] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:25.422861 19832 ts_tablet_manager.cc:1905] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:25.424510 19838 ts_tablet_manager.cc:1918] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:25.424544 19833 ts_tablet_manager.cc:1918] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:25.424546 19832 ts_tablet_manager.cc:1918] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:25.424572 19838 log.cc:1196] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/4c3751775cb14f5bae080558e4497b21
I20260808 20:56:25.424597 19833 log.cc:1196] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/4c3751775cb14f5bae080558e4497b21
I20260808 20:56:25.424597 19832 log.cc:1196] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/4c3751775cb14f5bae080558e4497b21
I20260808 20:56:25.424814 19838 ts_tablet_manager.cc:1939] T 4c3751775cb14f5bae080558e4497b21 P 7be5243b99494a19b523e2157a1118a8: Deleting consensus metadata
I20260808 20:56:25.424834 19832 ts_tablet_manager.cc:1939] T 4c3751775cb14f5bae080558e4497b21 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting consensus metadata
I20260808 20:56:25.424834 19833 ts_tablet_manager.cc:1939] T 4c3751775cb14f5bae080558e4497b21 P 12ff3ef0cf31433a928e08b40d506aac: Deleting consensus metadata
I20260808 20:56:25.425129 18896 catalog_manager.cc:4841] TS 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999): tablet 4c3751775cb14f5bae080558e4497b21 (table test-table [id=276b463ed9ef4623afbfa91beba1a77f]) successfully deleted
I20260808 20:56:25.425210 18896 catalog_manager.cc:4841] TS 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225): tablet 4c3751775cb14f5bae080558e4497b21 (table test-table [id=276b463ed9ef4623afbfa91beba1a77f]) successfully deleted
I20260808 20:56:25.425168 18922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 20:56:25.425209 18899 catalog_manager.cc:4841] TS 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947): tablet 4c3751775cb14f5bae080558e4497b21 (table test-table [id=276b463ed9ef4623afbfa91beba1a77f]) successfully deleted
W20260808 20:56:25.425328 18922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:25.426718 19338 tablet_service.cc:1467] Processing CreateTablet for tablet 08f8ae78df98411f9cd7fa7026859644 (DEFAULT_TABLE table=test-table [id=16e478441aa44f2288652bde87573e7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:25.426749 19564 tablet_service.cc:1467] Processing CreateTablet for tablet 08f8ae78df98411f9cd7fa7026859644 (DEFAULT_TABLE table=test-table [id=16e478441aa44f2288652bde87573e7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:25.426759 19128 tablet_service.cc:1467] Processing CreateTablet for tablet 08f8ae78df98411f9cd7fa7026859644 (DEFAULT_TABLE table=test-table [id=16e478441aa44f2288652bde87573e7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:25.426839 19338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f8ae78df98411f9cd7fa7026859644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:25.426838 19564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f8ae78df98411f9cd7fa7026859644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:25.426879 19128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f8ae78df98411f9cd7fa7026859644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:25.427868 19852 tablet_bootstrap.cc:492] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8: Bootstrap starting.
I20260808 20:56:25.428056 19853 tablet_bootstrap.cc:492] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Bootstrap starting.
I20260808 20:56:25.428073 19851 tablet_bootstrap.cc:492] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: Bootstrap starting.
I20260808 20:56:25.428359 19851 tablet_bootstrap.cc:654] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:25.428360 19852 tablet_bootstrap.cc:654] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:25.428471 19853 tablet_bootstrap.cc:654] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:25.428929 19851 tablet_bootstrap.cc:492] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: No bootstrap required, opened a new log
I20260808 20:56:25.428973 19851 ts_tablet_manager.cc:1397] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:25.429034 19853 tablet_bootstrap.cc:492] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: No bootstrap required, opened a new log
I20260808 20:56:25.429085 19853 ts_tablet_manager.cc:1397] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:25.429129 19851 raft_consensus.cc:348] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.429205 19851 raft_consensus.cc:374] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:25.429229 19851 raft_consensus.cc:729] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ff3ef0cf31433a928e08b40d506aac, State: Initialized, Role: FOLLOWER
I20260808 20:56:25.429201 19853 raft_consensus.cc:348] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.429271 19853 raft_consensus.cc:374] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:25.429308 19853 raft_consensus.cc:729] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a796dc5f7428797f4a6e23c5bb3f8, State: Initialized, Role: FOLLOWER
I20260808 20:56:25.429288 19851 consensus_queue.cc:260] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.429366 19853 consensus_queue.cc:260] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.429396 19851 ts_tablet_manager.cc:1428] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:25.429471 19853 ts_tablet_manager.cc:1428] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:25.430162 19852 tablet_bootstrap.cc:492] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8: No bootstrap required, opened a new log
I20260808 20:56:25.430219 19852 ts_tablet_manager.cc:1397] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:25.430334 19852 raft_consensus.cc:348] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.430400 19852 raft_consensus.cc:374] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:25.430424 19852 raft_consensus.cc:729] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be5243b99494a19b523e2157a1118a8, State: Initialized, Role: FOLLOWER
I20260808 20:56:25.430461 19852 consensus_queue.cc:260] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.430526 19852 ts_tablet_manager.cc:1428] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:25.549108 19823 raft_consensus.cc:482] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:25.549216 19823 raft_consensus.cc:504] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.549399 19823 leader_election.cc:288] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947), 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999)
I20260808 20:56:25.549696 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f8ae78df98411f9cd7fa7026859644" candidate_uuid: "12ff3ef0cf31433a928e08b40d506aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5243b99494a19b523e2157a1118a8" is_pre_election: true
I20260808 20:56:25.549755 19595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f8ae78df98411f9cd7fa7026859644" candidate_uuid: "12ff3ef0cf31433a928e08b40d506aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" is_pre_election: true
I20260808 20:56:25.549831 19139 raft_consensus.cc:2454] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ff3ef0cf31433a928e08b40d506aac in term 0.
I20260808 20:56:25.549844 19595 raft_consensus.cc:2454] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12ff3ef0cf31433a928e08b40d506aac in term 0.
I20260808 20:56:25.550031 19270 leader_election.cc:302] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ff3ef0cf31433a928e08b40d506aac, 7be5243b99494a19b523e2157a1118a8; no voters:
I20260808 20:56:25.550143 19823 raft_consensus.cc:2781] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:25.550197 19823 raft_consensus.cc:482] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:25.550228 19823 raft_consensus.cc:3037] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:25.550992 19823 raft_consensus.cc:504] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.551159 19823 leader_election.cc:288] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [CANDIDATE]: Term 1 election: Requested vote from peers 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947), 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999)
I20260808 20:56:25.551333 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f8ae78df98411f9cd7fa7026859644" candidate_uuid: "12ff3ef0cf31433a928e08b40d506aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5243b99494a19b523e2157a1118a8"
I20260808 20:56:25.551345 19595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f8ae78df98411f9cd7fa7026859644" candidate_uuid: "12ff3ef0cf31433a928e08b40d506aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8"
I20260808 20:56:25.551425 19139 raft_consensus.cc:3037] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:25.551424 19595 raft_consensus.cc:3037] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:25.552062 19139 raft_consensus.cc:2454] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ff3ef0cf31433a928e08b40d506aac in term 1.
I20260808 20:56:25.552217 19595 raft_consensus.cc:2454] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12ff3ef0cf31433a928e08b40d506aac in term 1.
I20260808 20:56:25.552320 19270 leader_election.cc:302] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ff3ef0cf31433a928e08b40d506aac, 7be5243b99494a19b523e2157a1118a8; no voters:
I20260808 20:56:25.552418 19823 raft_consensus.cc:2781] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:25.552527 19823 raft_consensus.cc:686] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 1 LEADER]: Becoming Leader. State: Replica: 12ff3ef0cf31433a928e08b40d506aac, State: Running, Role: LEADER
I20260808 20:56:25.552598 19823 consensus_queue.cc:237] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } }
I20260808 20:56:25.553220 18923 catalog_manager.cc:5495] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac reported cstate change: term changed from 0 to 1, leader changed from <none> to 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130). New cstate: current_term: 1 leader_uuid: "12ff3ef0cf31433a928e08b40d506aac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12ff3ef0cf31433a928e08b40d506aac" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 } health_report { overall_health: UNKNOWN } } }
I20260808 20:56:25.580569 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:25.578232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.15.232.130:45225)
0808 20:56:25.578462 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:25.578463 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:25.578470 (+ 7us) server_negotiation.cc:407] Connection header received
0808 20:56:25.578490 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:25.578491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:25.578501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:25.578575 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:25.578677 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:25.579153 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:25.579939 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:25.580136 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:25.580199 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:25.580307 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:25.580338 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:25.580378 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:25.580419 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 20:56:25.580442 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":140,"threads_started":1}
I20260808 20:56:25.583009 19139 raft_consensus.cc:1261] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8 [term 1 FOLLOWER]: Refusing update from remote peer 12ff3ef0cf31433a928e08b40d506aac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:25.583171 19595 raft_consensus.cc:1261] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Refusing update from remote peer 12ff3ef0cf31433a928e08b40d506aac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:25.583241 19823 consensus_queue.cc:1035] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be5243b99494a19b523e2157a1118a8" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 38947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:25.583361 19787 consensus_queue.cc:1035] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a796dc5f7428797f4a6e23c5bb3f8" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 20:56:25.588487 19807 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 20:56:27.109252 18923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:27.109381 18923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:27.110265 18923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bddb3b47089244ad941421fde55eabe7: Sending DeleteTablet for 3 replicas of tablet 08f8ae78df98411f9cd7fa7026859644
I20260808 20:56:27.110504 19128 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f8ae78df98411f9cd7fa7026859644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:27 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:48032
I20260808 20:56:27.110620 19564 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f8ae78df98411f9cd7fa7026859644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:27 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34854
I20260808 20:56:27.110703 19338 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f8ae78df98411f9cd7fa7026859644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:27 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:41860
I20260808 20:56:27.110728 19914 tablet_replica.cc:330] T 08f8ae78df98411f9cd7fa7026859644 P 7be5243b99494a19b523e2157a1118a8: stopping tablet replica
I20260808 20:56:27.110741 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 19030
I20260808 20:56:27.118571 19919 tablet_replica.cc:330] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: stopping tablet replica
I20260808 20:56:27.118702 19919 raft_consensus.cc:2229] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:27.118806 19919 raft_consensus.cc:2258] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:27.118963 19915 tablet_replica.cc:330] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: stopping tablet replica
I20260808 20:56:27.119042 19915 raft_consensus.cc:2229] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:27.119102 19915 raft_consensus.cc:2258] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:27.119374 19915 ts_tablet_manager.cc:1905] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:27.120966 19915 ts_tablet_manager.cc:1918] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 20:56:27.120957 19919 ts_tablet_manager.cc:1905] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:27.121034 19915 log.cc:1196] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/08f8ae78df98411f9cd7fa7026859644
I20260808 20:56:27.121263 19915 ts_tablet_manager.cc:1939] T 08f8ae78df98411f9cd7fa7026859644 P 5e6a796dc5f7428797f4a6e23c5bb3f8: Deleting consensus metadata
I20260808 20:56:27.121690 18896 catalog_manager.cc:4841] TS 5e6a796dc5f7428797f4a6e23c5bb3f8 (127.15.232.131:43999): tablet 08f8ae78df98411f9cd7fa7026859644 (table test-table [id=16e478441aa44f2288652bde87573e7d]) successfully deleted
I20260808 20:56:27.122527 19919 ts_tablet_manager.cc:1918] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 20:56:27.122606 19919 log.cc:1196] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/08f8ae78df98411f9cd7fa7026859644
I20260808 20:56:27.122879 19919 ts_tablet_manager.cc:1939] T 08f8ae78df98411f9cd7fa7026859644 P 12ff3ef0cf31433a928e08b40d506aac: Deleting consensus metadata
I20260808 20:56:27.123279 18896 catalog_manager.cc:4841] TS 12ff3ef0cf31433a928e08b40d506aac (127.15.232.130:45225): tablet 08f8ae78df98411f9cd7fa7026859644 (table test-table [id=16e478441aa44f2288652bde87573e7d]) successfully deleted
W20260808 20:56:27.129195 18899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:38947 (error 108)
I20260808 20:56:27.129478 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 19240
I20260808 20:56:27.135574 19923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:27.134868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:38947 (local address 127.0.0.1:39568)
0808 20:56:27.135428 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:27.135506 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:38947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":480,"threads_started":1}
W20260808 20:56:27.135687 18899 catalog_manager.cc:4584] TS 7be5243b99494a19b523e2157a1118a8 (127.15.232.129:38947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 08f8ae78df98411f9cd7fa7026859644: Network error: Client connection negotiation failed: client connection to 127.15.232.129:38947: connect: Connection refused (error 111)
I20260808 20:56:27.145303 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 19464
I20260808 20:56:27.157231 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 18873
2026-08-08T20:56:27Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5848 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:27 dist-test-slave-c12x krb5kdc[19926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:27 dist-test-slave-c12x krb5kdc[19926](info): set up 2 sockets
Aug 08 20:56:27 dist-test-slave-c12x krb5kdc[19926](info): commencing operation
krb5kdc: starting...
W20260808 20:56:28.405999 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:28 dist-test-slave-c12x krb5kdc[19926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:28.539871 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37341
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:37341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:28.630192 19949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:28.630491 19949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:28.630535 19949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:28.631880 19949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:28.631963 19949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:28.631989 19949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:28.632023 19949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:28.632055 19949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:28.633738 19949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:37341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:28.634058 19949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:28.634315 19949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:28.639722 19957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:28.639935 19963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:28.639922 19959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:28.640285 19949 server_base.cc:1034] running on GCE node
I20260808 20:56:28.640554 19949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:28.640873 19949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:28.642058 19949 hybrid_clock.cc:648] HybridClock initialized: now 1786222588642025 us; error 57 us; skew 500 ppm
Aug 08 20:56:28 dist-test-slave-c12x krb5kdc[19926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222588, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:28.645622 19949 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:28.646481 19949 webserver.cc:460] Webserver started at http://127.15.232.190:41725/ using document root <none> and password file <none>
I20260808 20:56:28.646801 19949 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:28.646884 19949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:28.647055 19949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:28.648411 19949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "c9e51e79e44f40769939cea5e41c7f58"
format_stamp: "Formatted at 2026-08-08 20:56:28 on dist-test-slave-c12x"
server_key: "2e1b9ab694313ea03fb752e9159d2845"
server_key_iv: "85098eb3590594a5e53151334080243b"
server_key_version: "encryptionkey@0"
I20260808 20:56:28.648816 19949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "c9e51e79e44f40769939cea5e41c7f58"
format_stamp: "Formatted at 2026-08-08 20:56:28 on dist-test-slave-c12x"
server_key: "2e1b9ab694313ea03fb752e9159d2845"
server_key_iv: "85098eb3590594a5e53151334080243b"
server_key_version: "encryptionkey@0"
I20260808 20:56:28.650291 19949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:28.651237 19972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:28.651559 19949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:28.651681 19949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "c9e51e79e44f40769939cea5e41c7f58"
format_stamp: "Formatted at 2026-08-08 20:56:28 on dist-test-slave-c12x"
server_key: "2e1b9ab694313ea03fb752e9159d2845"
server_key_iv: "85098eb3590594a5e53151334080243b"
server_key_version: "encryptionkey@0"
I20260808 20:56:28.651773 19949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:28.663698 19949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:28.666023 19949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:28.666217 19949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:28.670974 19949 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:37341
I20260808 20:56:28.671015 20058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:37341 every 8 connection(s)
I20260808 20:56:28.671411 19949 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:28.672451 20060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:28.674904 20060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58: Bootstrap starting.
I20260808 20:56:28.675606 20060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:28.675889 20060 log.cc:824] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:28.676631 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 19949
I20260808 20:56:28.676894 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:28.677071 20060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58: No bootstrap required, opened a new log
I20260808 20:56:28.677227 16290 external_mini_cluster.cc:1428] Setting key 0431b09cbe1b148a159d78c33fb7026f
I20260808 20:56:28.679641 20060 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e51e79e44f40769939cea5e41c7f58" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37341 } }
I20260808 20:56:28.679841 20060 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:28.679908 20060 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9e51e79e44f40769939cea5e41c7f58, State: Initialized, Role: FOLLOWER
I20260808 20:56:28.680075 20060 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e51e79e44f40769939cea5e41c7f58" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37341 } }
I20260808 20:56:28.680186 20060 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:28.680236 20060 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:28.680312 20060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:28.681411 20060 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e51e79e44f40769939cea5e41c7f58" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37341 } }
I20260808 20:56:28.681622 20060 leader_election.cc:302] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9e51e79e44f40769939cea5e41c7f58; no voters:
I20260808 20:56:28.681866 20060 leader_election.cc:288] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:28.681898 20067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:28.682025 20067 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [term 1 LEADER]: Becoming Leader. State: Replica: c9e51e79e44f40769939cea5e41c7f58, State: Running, Role: LEADER
I20260808 20:56:28.682181 20067 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e51e79e44f40769939cea5e41c7f58" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37341 } }
I20260808 20:56:28.682215 20060 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:28.682870 20068 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9e51e79e44f40769939cea5e41c7f58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e51e79e44f40769939cea5e41c7f58" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37341 } } }
I20260808 20:56:28.683418 20068 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:28.683318 20069 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9e51e79e44f40769939cea5e41c7f58. Latest consensus state: current_term: 1 leader_uuid: "c9e51e79e44f40769939cea5e41c7f58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e51e79e44f40769939cea5e41c7f58" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37341 } } }
I20260808 20:56:28.683744 20069 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58 [sys.catalog]: This master's current role is: LEADER
W20260808 20:56:28.684556 20092 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 20:56:28.684620 20092 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 20:56:28.684705 20083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:28.684894 20083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 20:56:28 dist-test-slave-c12x krb5kdc[19926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:28.688064 20083 catalog_manager.cc:1347] Generated new cluster ID: 7ee7e13a3ac74e2d9362f5de3e6e0f77
I20260808 20:56:28.688133 20083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:28.692416 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:28.678032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.15.232.190:37341)
0808 20:56:28.678256 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:28.678264 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:28.678292 (+ 28us) server_negotiation.cc:407] Connection header received
0808 20:56:28.679639 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:28.679648 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:28.679713 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:28.679896 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:28.680765 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:28.682471 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:28.683670 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:28.684385 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:28.684495 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:28.687719 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:28.687725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:28.687726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:28.687742 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:28.689976 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:28.690409 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:28.690413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:28.690414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:28.690566 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:28.690781 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:28.690782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:28.690783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:28.691206 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:28.691600 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:28.691976 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 20:56:28.692158 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":108,"threads_started":1}
I20260808 20:56:28.696208 20083 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:28.697211 20083 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:28.701730 20083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9e51e79e44f40769939cea5e41c7f58: Generated new TSK 0
I20260808 20:56:28.701946 20083 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:28.730621 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37341
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:28.821889 20100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:28.822141 20100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:28.822182 20100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:28.823525 20100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:28.823590 20100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:28.823635 20100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:28.825201 20100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:28.825485 20100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:28.825736 20100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:28.831175 20111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:28.831175 20113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:28.831526 20100 server_base.cc:1034] running on GCE node
W20260808 20:56:28.831221 20110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:28.831871 20100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:28.832159 20100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:28.833334 20100 hybrid_clock.cc:648] HybridClock initialized: now 1786222588833310 us; error 39 us; skew 500 ppm
Aug 08 20:56:28 dist-test-slave-c12x krb5kdc[19926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222588, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:28.836059 20100 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:28.836565 20100 webserver.cc:460] Webserver started at http://127.15.232.129:38473/ using document root <none> and password file <none>
I20260808 20:56:28.836808 20100 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:28.836880 20100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:28.837033 20100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:28.838168 20100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "d985cbe6213e4df68d5e1f582c816e19"
format_stamp: "Formatted at 2026-08-08 20:56:28 on dist-test-slave-c12x"
server_key: "9db678cffc33e6224282a98d64109dd4"
server_key_iv: "e073ffa92b3f5392fc34a6518b81bfe9"
server_key_version: "encryptionkey@0"
I20260808 20:56:28.838569 20100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "d985cbe6213e4df68d5e1f582c816e19"
format_stamp: "Formatted at 2026-08-08 20:56:28 on dist-test-slave-c12x"
server_key: "9db678cffc33e6224282a98d64109dd4"
server_key_iv: "e073ffa92b3f5392fc34a6518b81bfe9"
server_key_version: "encryptionkey@0"
I20260808 20:56:28.840127 20100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:28.840992 20122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:28.841236 20100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:28.841352 20100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "d985cbe6213e4df68d5e1f582c816e19"
format_stamp: "Formatted at 2026-08-08 20:56:28 on dist-test-slave-c12x"
server_key: "9db678cffc33e6224282a98d64109dd4"
server_key_iv: "e073ffa92b3f5392fc34a6518b81bfe9"
server_key_version: "encryptionkey@0"
I20260808 20:56:28.841434 20100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:28.849938 20100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:28.852679 20100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:28.852864 20100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:28.853142 20100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:28.853536 20100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:28.853600 20100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:28.853648 20100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:28.853696 20100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:28.859890 20100 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:33765
I20260808 20:56:28.859956 20293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:33765 every 8 connection(s)
I20260808 20:56:28.860327 20100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
Aug 08 20:56:28 dist-test-slave-c12x krb5kdc[19926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222588, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:28.867609 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 20100
I20260808 20:56:28.867717 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:28.867913 16290 external_mini_cluster.cc:1428] Setting key b79c52e5d619cc0868a883a74e3ab7fe
I20260808 20:56:28.870822 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:28.861175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:34103 (local address 127.15.232.190:37341)
0808 20:56:28.861248 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:28.861251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:28.861460 (+ 209us) server_negotiation.cc:407] Connection header received
0808 20:56:28.861964 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:28.861965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:28.861978 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:28.862053 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:28.862982 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:28.863577 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:28.864654 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:28.864881 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:28.864954 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:28.867359 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:28.867360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:28.867361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:28.867368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:28.869039 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:28.869564 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:28.869565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:28.869565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:28.869619 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:28.869931 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:28.869932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:28.869932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:28.870063 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:28.870135 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:28.870558 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 20:56:28.870683 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 20:56:28.870846 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:28.861164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37341 (local address 127.15.232.129:34103)
0808 20:56:28.861337 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:28.861374 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:28.861870 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:28.862108 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:28.862112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:28.862694 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:28.862893 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:28.862898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:28.863687 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:28.863688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:28.864549 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:28.864552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:28.864994 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:28.864995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:28.865121 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:28.865665 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:28.865675 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:28.867256 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:28.869131 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:28.869139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:28.869142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:28.869482 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:28.869690 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:28.869691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:28.869692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:28.869870 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:28.870141 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:28.870148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:28.870327 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 20:56:28.870455 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 20:56:28.870656 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260808 20:56:28.871486 20295 heartbeater.cc:347] Connected to a master server at 127.15.232.190:37341
I20260808 20:56:28.871646 20295 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:28.872023 20295 heartbeater.cc:511] Master 127.15.232.190:37341 requested a full tablet report, sending...
I20260808 20:56:28.872603 20000 ts_manager.cc:194] Registered new tserver with Master: d985cbe6213e4df68d5e1f582c816e19 (127.15.232.129:33765)
I20260808 20:56:28.873253 20000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:34103
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:28.898455 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37341
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:28.984010 20301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:28.984258 20301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:28.984300 20301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:28.985597 20301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:28.985668 20301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:28.985713 20301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:28.987255 20301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:28.987572 20301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:28.987821 20301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:28.993360 20308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:28.993549 20301 server_base.cc:1034] running on GCE node
W20260808 20:56:28.993346 20310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:28.993350 20307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:28.993916 20301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:28.994200 20301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:28.995354 20301 hybrid_clock.cc:648] HybridClock initialized: now 1786222588995334 us; error 40 us; skew 500 ppm
Aug 08 20:56:28 dist-test-slave-c12x krb5kdc[19926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222588, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:28.998077 20301 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:28.998590 20301 webserver.cc:460] Webserver started at http://127.15.232.130:39777/ using document root <none> and password file <none>
I20260808 20:56:28.998934 20301 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:28.999008 20301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:28.999150 20301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:29.000373 20301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "07a61fa0077b4c34b70ced4740843b7a"
format_stamp: "Formatted at 2026-08-08 20:56:28 on dist-test-slave-c12x"
server_key: "25570120773dc31c688e142317fcfc66"
server_key_iv: "450829a361b1169e9d60e9ccb9617183"
server_key_version: "encryptionkey@0"
I20260808 20:56:29.000895 20301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "07a61fa0077b4c34b70ced4740843b7a"
format_stamp: "Formatted at 2026-08-08 20:56:28 on dist-test-slave-c12x"
server_key: "25570120773dc31c688e142317fcfc66"
server_key_iv: "450829a361b1169e9d60e9ccb9617183"
server_key_version: "encryptionkey@0"
I20260808 20:56:29.002384 20301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:29.003252 20321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:29.003544 20301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:29.003649 20301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "07a61fa0077b4c34b70ced4740843b7a"
format_stamp: "Formatted at 2026-08-08 20:56:28 on dist-test-slave-c12x"
server_key: "25570120773dc31c688e142317fcfc66"
server_key_iv: "450829a361b1169e9d60e9ccb9617183"
server_key_version: "encryptionkey@0"
I20260808 20:56:29.003733 20301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:29.021394 20301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:29.023607 20301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:29.023773 20301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:29.024044 20301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:29.024395 20301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:29.024438 20301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:29.024487 20301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:29.024518 20301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:29.030712 20301 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:33593
I20260808 20:56:29.030764 20488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:33593 every 8 connection(s)
I20260808 20:56:29.031165 20301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:29.034785 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 20301
I20260808 20:56:29.034909 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:29.035094 16290 external_mini_cluster.cc:1428] Setting key 0f7d2b0a5d17e93642a43e093dd6d64c
Aug 08 20:56:29 dist-test-slave-c12x krb5kdc[19926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222588, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:29.041126 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:29.031957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:45367 (local address 127.15.232.190:37341)
0808 20:56:29.032016 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:29.032018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:29.032210 (+ 192us) server_negotiation.cc:407] Connection header received
0808 20:56:29.032697 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:29.032699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:29.032709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:29.032771 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:29.033713 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:29.034220 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:29.035335 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:29.035617 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:29.035676 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:29.038079 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:29.038081 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:29.038081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:29.038090 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:29.039660 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:29.040100 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:29.040102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:29.040102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:29.040166 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:29.040429 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:29.040430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:29.040431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:29.040559 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:29.040634 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:29.040926 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 20:56:29.041032 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:56:29.041329 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:29.031901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37341 (local address 127.15.232.130:45367)
0808 20:56:29.032073 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:29.032116 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:29.032598 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:29.032826 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:29.032829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:29.033409 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:29.033612 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:29.033617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:29.034307 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:29.034308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:29.035228 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:29.035235 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:29.035707 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:29.035708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:29.035863 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:29.036479 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:29.036494 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:29.037971 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:29.039758 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:29.039765 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:29.039768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:29.040010 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:29.040241 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:29.040241 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:29.040242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:29.040358 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:29.040635 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:29.040643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:29.040790 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 20:56:29.040904 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 20:56:29.041078 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260808 20:56:29.041780 20489 heartbeater.cc:347] Connected to a master server at 127.15.232.190:37341
I20260808 20:56:29.041904 20489 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:29.042104 20489 heartbeater.cc:511] Master 127.15.232.190:37341 requested a full tablet report, sending...
I20260808 20:56:29.042498 20000 ts_manager.cc:194] Registered new tserver with Master: 07a61fa0077b4c34b70ced4740843b7a (127.15.232.130:33593)
I20260808 20:56:29.042898 20000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:45367
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:29.066234 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37341
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:29.152395 20496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:29.152608 20496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:29.152633 20496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:29.153815 20496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:29.153860 20496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:29.153887 20496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:29.155651 20496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:29.155929 20496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:29.156183 20496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:29.161415 20502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:29.161499 20505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:29.161399 20501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:29.161753 20496 server_base.cc:1034] running on GCE node
I20260808 20:56:29.161971 20496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:29.162178 20496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:29.163324 20496 hybrid_clock.cc:648] HybridClock initialized: now 1786222589163313 us; error 32 us; skew 500 ppm
Aug 08 20:56:29 dist-test-slave-c12x krb5kdc[19926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222589, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:29.166081 20496 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:29.166587 20496 webserver.cc:460] Webserver started at http://127.15.232.131:45791/ using document root <none> and password file <none>
I20260808 20:56:29.166831 20496 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:29.166900 20496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:29.167061 20496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:29.168272 20496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "b0bb443801ef4085837489dae4dbc6db"
format_stamp: "Formatted at 2026-08-08 20:56:29 on dist-test-slave-c12x"
server_key: "500e943e72f658999d8db6daf8efc59a"
server_key_iv: "4910f35bb75595b6fb977fe0f4baac8a"
server_key_version: "encryptionkey@0"
I20260808 20:56:29.168666 20496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "b0bb443801ef4085837489dae4dbc6db"
format_stamp: "Formatted at 2026-08-08 20:56:29 on dist-test-slave-c12x"
server_key: "500e943e72f658999d8db6daf8efc59a"
server_key_iv: "4910f35bb75595b6fb977fe0f4baac8a"
server_key_version: "encryptionkey@0"
I20260808 20:56:29.170065 20496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 20:56:29.170951 20513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:29.171178 20496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:29.171293 20496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "b0bb443801ef4085837489dae4dbc6db"
format_stamp: "Formatted at 2026-08-08 20:56:29 on dist-test-slave-c12x"
server_key: "500e943e72f658999d8db6daf8efc59a"
server_key_iv: "4910f35bb75595b6fb977fe0f4baac8a"
server_key_version: "encryptionkey@0"
I20260808 20:56:29.171372 20496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:29.186264 20496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:29.188820 20496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:29.189002 20496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:29.189285 20496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:29.189682 20496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:29.189738 20496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:29.189781 20496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:29.189808 20496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:29.196274 20496 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:43687
I20260808 20:56:29.196318 20696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:43687 every 8 connection(s)
I20260808 20:56:29.196691 20496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:29.202428 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 20496
I20260808 20:56:29.202517 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:29.202726 16290 external_mini_cluster.cc:1428] Setting key 7a24be1458dc72b3b7a79cf0d2c5efb0
Aug 08 20:56:29 dist-test-slave-c12x krb5kdc[19926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222589, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:29.207837 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:29.197544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:58813 (local address 127.15.232.190:37341)
0808 20:56:29.197635 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:29.197638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:29.197853 (+ 215us) server_negotiation.cc:407] Connection header received
0808 20:56:29.198482 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:29.198483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:29.198495 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:29.198552 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:29.199597 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:29.200189 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:29.201222 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:29.201432 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:29.201487 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:29.204248 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:29.204250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:29.204250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:29.204257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:29.205855 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:29.206311 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:29.206312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:29.206312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:29.206373 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:29.206867 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:29.206869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:29.206869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:29.207075 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:29.207159 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:29.207609 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 20:56:29.207716 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 20:56:29.208065 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:29.197510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37341 (local address 127.15.232.131:58813)
0808 20:56:29.197701 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:29.197745 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:29.198364 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:29.198608 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:29.198611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:29.199223 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:29.199423 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:29.199428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:29.200289 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:29.200290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:29.201140 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:29.201142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:29.201545 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:29.201546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:29.201702 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:29.202341 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:29.202350 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:29.204161 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:29.205959 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:29.205966 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:29.205968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:29.206226 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:29.206553 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:29.206554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:29.206555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:29.206749 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:29.207157 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:29.207163 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:29.207368 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 20:56:29.207555 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 20:56:29.207764 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260808 20:56:29.208673 20697 heartbeater.cc:347] Connected to a master server at 127.15.232.190:37341
I20260808 20:56:29.208772 20697 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:29.208925 20697 heartbeater.cc:511] Master 127.15.232.190:37341 requested a full tablet report, sending...
I20260808 20:56:29.209333 20000 ts_manager.cc:194] Registered new tserver with Master: b0bb443801ef4085837489dae4dbc6db (127.15.232.131:43687)
I20260808 20:56:29.209736 20000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:58813
I20260808 20:56:29.214717 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 20:56:29.228730 20066 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:29.227611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.15.232.190:37341)
0808 20:56:29.227692 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:29.227694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:29.227870 (+ 176us) server_negotiation.cc:407] Connection header received
0808 20:56:29.227963 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:29.227964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:29.227977 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:29.228047 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:29.228649 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50452: BlockingRecv error: recv got EOF from 127.0.0.1:50452 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 20:56:29.229662 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 20100
I20260808 20:56:29.244367 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 20301
I20260808 20:56:29.253104 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 20496
I20260808 20:56:29.261813 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 19949
2026-08-08T20:56:29Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2101 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:29 dist-test-slave-c12x krb5kdc[20721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:29 dist-test-slave-c12x krb5kdc[20721](info): set up 2 sockets
Aug 08 20:56:29 dist-test-slave-c12x krb5kdc[20721](info): commencing operation
krb5kdc: starting...
W20260808 20:56:30.537921 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260808 20:56:30.721491 16290 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.183s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260808 20:56:30.838236 16290 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.117s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260808 20:56:30.971798 16290 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.133s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/test-user.keytab.
W20260808 20:56:31.373634 16290 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.402s user 0.000s sys 0.002s
Aug 08 20:56:31 dist-test-slave-c12x krb5kdc[20721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:31.426055 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:41119
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:41119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:31.512420 20744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:31.512656 20744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:31.512681 20744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:31.513839 20744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:31.513885 20744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:31.513899 20744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:31.513911 20744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:31.513924 20744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:31.515547 20744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:41119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:41119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:31.515861 20744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:31.516131 20744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:31.521637 20752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:31.521705 20750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:31.521644 20749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:31.522318 20744 server_base.cc:1034] running on GCE node
I20260808 20:56:31.522552 20744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:31.522833 20744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:31.524000 20744 hybrid_clock.cc:648] HybridClock initialized: now 1786222591523970 us; error 45 us; skew 500 ppm
Aug 08 20:56:31 dist-test-slave-c12x krb5kdc[20721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222591, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:31.529764 20744 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:31.530205 20744 webserver.cc:460] Webserver started at http://127.15.232.190:38627/ using document root <none> and password file <none>
I20260808 20:56:31.530426 20744 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:31.530469 20744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:31.530577 20744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:31.531816 20744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6"
format_stamp: "Formatted at 2026-08-08 20:56:31 on dist-test-slave-c12x"
server_key: "5c299860e8b0e57c4d18f0eee54db8b1"
server_key_iv: "5da8c718b2664d761062d474555aa8d4"
server_key_version: "encryptionkey@0"
I20260808 20:56:31.532223 20744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6"
format_stamp: "Formatted at 2026-08-08 20:56:31 on dist-test-slave-c12x"
server_key: "5c299860e8b0e57c4d18f0eee54db8b1"
server_key_iv: "5da8c718b2664d761062d474555aa8d4"
server_key_version: "encryptionkey@0"
I20260808 20:56:31.533740 20744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:31.534555 20767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:31.534780 20744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:31.534878 20744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6"
format_stamp: "Formatted at 2026-08-08 20:56:31 on dist-test-slave-c12x"
server_key: "5c299860e8b0e57c4d18f0eee54db8b1"
server_key_iv: "5da8c718b2664d761062d474555aa8d4"
server_key_version: "encryptionkey@0"
I20260808 20:56:31.534938 20744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:31.547207 20744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:31.549635 20744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:31.549757 20744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:31.554196 20744 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:41119
I20260808 20:56:31.554281 20844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:41119 every 8 connection(s)
I20260808 20:56:31.554827 20744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:31.556069 20845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:31.559126 20845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6: Bootstrap starting.
I20260808 20:56:31.559873 20845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:31.560209 20845 log.cc:824] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:31.560972 20845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6: No bootstrap required, opened a new log
I20260808 20:56:31.562304 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 20744
I20260808 20:56:31.562428 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:31.562619 16290 external_mini_cluster.cc:1428] Setting key 7603b24ac29acf566732dac4cf67929b
I20260808 20:56:31.563043 20845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41119 } }
I20260808 20:56:31.563249 20845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:31.563335 20845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bbaee65cad9474a9f9ea14be2ec2ae6, State: Initialized, Role: FOLLOWER
I20260808 20:56:31.563534 20845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41119 } }
I20260808 20:56:31.563658 20845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:31.563702 20845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:31.563768 20845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:31.564414 20845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41119 } }
I20260808 20:56:31.564543 20845 leader_election.cc:302] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bbaee65cad9474a9f9ea14be2ec2ae6; no voters:
I20260808 20:56:31.564755 20845 leader_election.cc:288] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:31.564785 20852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:31.564903 20852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [term 1 LEADER]: Becoming Leader. State: Replica: 4bbaee65cad9474a9f9ea14be2ec2ae6, State: Running, Role: LEADER
I20260808 20:56:31.564987 20852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41119 } }
I20260808 20:56:31.565043 20845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:31.565296 20852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41119 } } }
I20260808 20:56:31.565380 20852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:31.565513 20852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bbaee65cad9474a9f9ea14be2ec2ae6. Latest consensus state: current_term: 1 leader_uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbaee65cad9474a9f9ea14be2ec2ae6" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41119 } } }
I20260808 20:56:31.565635 20860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:31.565662 20852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:31.566224 20860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:31.567981 20860 catalog_manager.cc:1347] Generated new cluster ID: 5f3dc984358e482ba17e458c7d51b854
I20260808 20:56:31.568041 20860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 20:56:31 dist-test-slave-c12x krb5kdc[20721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:31.575755 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:31.563200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45548 (local address 127.15.232.190:41119)
0808 20:56:31.564060 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:31.564068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:31.564097 (+ 29us) server_negotiation.cc:407] Connection header received
0808 20:56:31.564199 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:31.564204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:31.564246 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:31.564398 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:31.565267 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:31.566817 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:31.567946 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:31.568580 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:31.568696 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:31.571710 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:31.571714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:31.571715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:31.571727 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:31.573705 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:31.574177 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:31.574181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:31.574181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:31.574307 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:31.574542 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:31.574543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:31.574543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:31.574996 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:31.575201 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:31.575306 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 20:56:31.575469 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":762,"threads_started":1}
I20260808 20:56:31.576709 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 20:56:31.588133 20860 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:31.588927 20860 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:31.591859 20860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bbaee65cad9474a9f9ea14be2ec2ae6: Generated new TSK 0
I20260808 20:56:31.592067 20860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 20:56:31.598449 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:31.594555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45564 (local address 127.15.232.190:41119)
0808 20:56:31.594603 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:31.594611 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:31.594689 (+ 78us) server_negotiation.cc:407] Connection header received
0808 20:56:31.594764 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:31.594765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:31.594776 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:31.594828 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:31.595356 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:31.595908 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:31.597018 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:31.597232 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:31.597288 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:31.598001 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:31.598003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:31.598003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:31.598204 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:31.598220 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:31.598226 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:31.598274 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:31.598362 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 20:56:31.598387 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 20:56:31.601675 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 20744
2026-08-08T20:56:31Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2342 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:31 dist-test-slave-c12x krb5kdc[20900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:31 dist-test-slave-c12x krb5kdc[20900](info): set up 2 sockets
Aug 08 20:56:31 dist-test-slave-c12x krb5kdc[20900](info): commencing operation
krb5kdc: starting...
W20260808 20:56:32.835377 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:32 dist-test-slave-c12x krb5kdc[20900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:32.964843 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36583
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:36583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:33.051328 20917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:33.051620 20917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:33.051663 20917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:33.052886 20917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:33.052970 20917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:33.052997 20917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:33.053020 20917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:33.053048 20917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:33.054677 20917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:36583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:33.055119 20917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:33.055399 20917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:33.060566 20923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:33.060595 20925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:33.060622 20929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:33.061314 20917 server_base.cc:1034] running on GCE node
I20260808 20:56:33.061672 20917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:33.062074 20917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:33.063246 20917 hybrid_clock.cc:648] HybridClock initialized: now 1786222593063231 us; error 30 us; skew 500 ppm
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:33.066288 20917 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:33.066816 20917 webserver.cc:460] Webserver started at http://127.15.232.190:44905/ using document root <none> and password file <none>
I20260808 20:56:33.067075 20917 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:33.067145 20917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:33.067334 20917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:33.068567 20917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "57991c6cf18d43ffb9883ee2994f6838"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "e1249b244662812c4492cfbb2d8b427f"
server_key_iv: "b08e4e2ef7e47775fcb3469920a2292d"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.068979 20917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "57991c6cf18d43ffb9883ee2994f6838"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "e1249b244662812c4492cfbb2d8b427f"
server_key_iv: "b08e4e2ef7e47775fcb3469920a2292d"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.070549 20917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 20:56:33.071583 20939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.071870 20917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:33.071961 20917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "57991c6cf18d43ffb9883ee2994f6838"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "e1249b244662812c4492cfbb2d8b427f"
server_key_iv: "b08e4e2ef7e47775fcb3469920a2292d"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.072074 20917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:33.091708 20917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:33.093855 20917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:33.094033 20917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:33.098729 21014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:36583 every 8 connection(s)
I20260808 20:56:33.098771 20917 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:36583
I20260808 20:56:33.099330 20917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:33.100764 21015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:33.100996 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 20917
I20260808 20:56:33.101118 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:33.101323 16290 external_mini_cluster.cc:1428] Setting key cb0eb10e6c48ab066eb8e59107a16855
I20260808 20:56:33.104053 21015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838: Bootstrap starting.
I20260808 20:56:33.104817 21015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:33.105114 21015 log.cc:824] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:33.105887 21015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838: No bootstrap required, opened a new log
I20260808 20:56:33.108171 21015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57991c6cf18d43ffb9883ee2994f6838" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36583 } }
I20260808 20:56:33.108349 21015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:33.108387 21015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57991c6cf18d43ffb9883ee2994f6838, State: Initialized, Role: FOLLOWER
I20260808 20:56:33.108522 21015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57991c6cf18d43ffb9883ee2994f6838" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36583 } }
I20260808 20:56:33.108635 21015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:33.108672 21015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:33.108732 21015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:33.109591 21015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57991c6cf18d43ffb9883ee2994f6838" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36583 } }
I20260808 20:56:33.109699 21015 leader_election.cc:302] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57991c6cf18d43ffb9883ee2994f6838; no voters:
I20260808 20:56:33.109899 21015 leader_election.cc:288] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:33.109951 21020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:33.110137 21020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [term 1 LEADER]: Becoming Leader. State: Replica: 57991c6cf18d43ffb9883ee2994f6838, State: Running, Role: LEADER
I20260808 20:56:33.110244 21015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:33.110242 21020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57991c6cf18d43ffb9883ee2994f6838" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36583 } }
I20260808 20:56:33.110739 21020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57991c6cf18d43ffb9883ee2994f6838. Latest consensus state: current_term: 1 leader_uuid: "57991c6cf18d43ffb9883ee2994f6838" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57991c6cf18d43ffb9883ee2994f6838" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36583 } } }
I20260808 20:56:33.110750 21021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57991c6cf18d43ffb9883ee2994f6838" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57991c6cf18d43ffb9883ee2994f6838" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36583 } } }
I20260808 20:56:33.110894 21020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:33.111793 21037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:33.112303 21037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:33.112440 21021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:33.114218 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:33.101965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57746 (local address 127.15.232.190:36583)
0808 20:56:33.102209 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:33.102219 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:33.102255 (+ 36us) server_negotiation.cc:407] Connection header received
0808 20:56:33.102795 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:33.102804 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:33.102867 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:33.103028 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:33.103741 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.105137 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.106130 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.106740 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.106864 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:33.109612 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:33.109616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:33.109616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:33.109627 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:33.112269 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.112754 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.112757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.112758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.112865 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.113098 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.113099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.113100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.113416 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:33.113555 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:33.113856 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 20:56:33.114016 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
I20260808 20:56:33.114643 21037 catalog_manager.cc:1347] Generated new cluster ID: 6813d7de95d743a9acd7ce88ae3132a9
I20260808 20:56:33.114696 21037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:33.132946 21037 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:33.133715 21037 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:33.138079 21037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57991c6cf18d43ffb9883ee2994f6838: Generated new TSK 0
I20260808 20:56:33.138280 21037 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:33.174089 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36583
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:33.262858 21056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:33.263110 21056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:33.263154 21056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:33.264436 21056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:33.264508 21056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:33.264557 21056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:33.266126 21056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:33.266415 21056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:33.266670 21056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:33.272214 21063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:33.272217 21062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:33.272197 21065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:33.272748 21056 server_base.cc:1034] running on GCE node
I20260808 20:56:33.272979 21056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:33.273197 21056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:33.274391 21056 hybrid_clock.cc:648] HybridClock initialized: now 1786222593274378 us; error 34 us; skew 500 ppm
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:33.277366 21056 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:33.277909 21056 webserver.cc:460] Webserver started at http://127.15.232.129:36645/ using document root <none> and password file <none>
I20260808 20:56:33.278172 21056 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:33.278240 21056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:33.278386 21056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:33.279572 21056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "dcf90ae7cb874e7e9422e8cb5102e4ba"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "8d5d509613157e1e492ec4d5c1f8d99e"
server_key_iv: "0cdcc577bb0f4ed9383dd24650f173b2"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.279953 21056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "dcf90ae7cb874e7e9422e8cb5102e4ba"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "8d5d509613157e1e492ec4d5c1f8d99e"
server_key_iv: "0cdcc577bb0f4ed9383dd24650f173b2"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.281418 21056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 20:56:33.282370 21073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.282601 21056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:33.282730 21056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "dcf90ae7cb874e7e9422e8cb5102e4ba"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "8d5d509613157e1e492ec4d5c1f8d99e"
server_key_iv: "0cdcc577bb0f4ed9383dd24650f173b2"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.282814 21056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:33.308598 21056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:33.311477 21056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:33.311669 21056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:33.311935 21056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:33.312336 21056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:33.312389 21056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.312419 21056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:33.312434 21056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.319408 21056 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:39779
I20260808 20:56:33.319514 21241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:39779 every 8 connection(s)
I20260808 20:56:33.319886 21056 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:33.321043 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 21056
I20260808 20:56:33.321172 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:33.321374 16290 external_mini_cluster.cc:1428] Setting key a7777abc393f54346304eeffebd2f3b4
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
I20260808 20:56:33.331935 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:33.320854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:38111 (local address 127.15.232.190:36583)
0808 20:56:33.320938 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:33.320941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:33.321154 (+ 213us) server_negotiation.cc:407] Connection header received
0808 20:56:33.321834 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:33.321835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:33.321846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:33.321902 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:33.323117 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.323936 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.325512 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.325906 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.325986 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:33.328376 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:33.328378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:33.328379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:33.328389 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:33.330112 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.330610 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.330611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.330612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.330678 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.331067 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.331068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.331068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.331259 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:33.331346 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:33.331732 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 20:56:33.331836 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 20:56:33.332228 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:33.320807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36583 (local address 127.15.232.129:38111)
0808 20:56:33.321010 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:33.321054 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:33.321731 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:33.322069 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:33.322074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:33.322781 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:33.322995 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:33.323000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.324069 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.324071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:33.325414 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:33.325417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.326001 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.326002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:33.326130 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:33.326739 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:33.326748 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:33.328268 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:33.330234 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:33.330243 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:33.330246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:33.330513 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:33.330751 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:33.330752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:33.330753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:33.330869 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:33.331358 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:33.331366 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:33.331583 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 20:56:33.331706 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 20:56:33.331897 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":121,"threads_started":1}
I20260808 20:56:33.332834 21242 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36583
I20260808 20:56:33.332975 21242 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:33.333199 21242 heartbeater.cc:511] Master 127.15.232.190:36583 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
I20260808 20:56:33.334100 20964 ts_manager.cc:194] Registered new tserver with Master: dcf90ae7cb874e7e9422e8cb5102e4ba (127.15.232.129:39779)
I20260808 20:56:33.334885 20964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:38111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:33.351169 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36583
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:33.439234 21248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:33.439478 21248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:33.439505 21248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:33.440709 21248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:33.440755 21248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:33.440783 21248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:33.442309 21248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:33.442576 21248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:33.442821 21248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:33.448303 21255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:33.448346 21259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:33.448446 21257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:33.449326 21248 server_base.cc:1034] running on GCE node
I20260808 20:56:33.449549 21248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:33.449838 21248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:33.451023 21248 hybrid_clock.cc:648] HybridClock initialized: now 1786222593450998 us; error 43 us; skew 500 ppm
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:33.453864 21248 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:33.454303 21248 webserver.cc:460] Webserver started at http://127.15.232.130:33679/ using document root <none> and password file <none>
I20260808 20:56:33.454504 21248 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:33.454547 21248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:33.454643 21248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:33.455790 21248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "c09784fa2ac646b48c46b5b9bd1ab151"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "e71ebb8ce20047036933466fadee8347"
server_key_iv: "f314f0fc543e4c7d5bca6ebb2ba95f8f"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.456115 21248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "c09784fa2ac646b48c46b5b9bd1ab151"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "e71ebb8ce20047036933466fadee8347"
server_key_iv: "f314f0fc543e4c7d5bca6ebb2ba95f8f"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.457577 21248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:33.458331 21269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.458546 21248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:33.458624 21248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "c09784fa2ac646b48c46b5b9bd1ab151"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "e71ebb8ce20047036933466fadee8347"
server_key_iv: "f314f0fc543e4c7d5bca6ebb2ba95f8f"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.458681 21248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:33.475396 21248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:33.477542 21248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:33.477718 21248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:33.477982 21248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:33.478363 21248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:33.478399 21248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.478425 21248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:33.478468 21248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.484758 21248 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:45491
I20260808 20:56:33.484802 21410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:45491 every 8 connection(s)
I20260808 20:56:33.485179 21248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:33.488009 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 21248
I20260808 20:56:33.488142 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:33.488338 16290 external_mini_cluster.cc:1428] Setting key cd3491a6c82a6d2943196c4587c4a96d
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:33.496434 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:33.486018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:54109 (local address 127.15.232.190:36583)
0808 20:56:33.486078 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:33.486081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:33.486287 (+ 206us) server_negotiation.cc:407] Connection header received
0808 20:56:33.486844 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:33.486845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:33.486859 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:33.486909 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:33.488001 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.488752 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.489895 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.490109 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.490186 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:33.492957 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:33.492959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:33.492959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:33.492969 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:33.494800 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.495263 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.495264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.495265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.495339 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.495659 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.495660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.495661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.495838 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:33.495921 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:33.496228 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 20:56:33.496328 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 20:56:33.496466 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:33.485950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36583 (local address 127.15.232.130:54109)
0808 20:56:33.486149 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:33.486194 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:33.486727 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:33.486986 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:33.486990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:33.487684 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:33.487887 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:33.487892 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.488846 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.488848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:33.489788 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:33.489791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.490192 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.490194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:33.490331 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:33.490986 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:33.490998 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:33.492831 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:33.494901 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:33.494908 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:33.494911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:33.495177 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:33.495417 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:33.495417 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:33.495418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:33.495585 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:33.495920 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:33.495925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:33.496069 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 20:56:33.496154 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 20:56:33.496292 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260808 20:56:33.497036 21414 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36583
I20260808 20:56:33.497155 21414 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:33.497368 21414 heartbeater.cc:511] Master 127.15.232.190:36583 requested a full tablet report, sending...
I20260808 20:56:33.497829 20964 ts_manager.cc:194] Registered new tserver with Master: c09784fa2ac646b48c46b5b9bd1ab151 (127.15.232.130:45491)
I20260808 20:56:33.498266 20964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:54109
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:33.518833 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36583
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:33.608095 21420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:33.608389 21420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:33.608436 21420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:33.609787 21420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:33.609877 21420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:33.609933 21420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:33.611945 21420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:33.612262 21420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:33.612535 21420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:33.620282 21428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:33.620522 21420 server_base.cc:1034] running on GCE node
W20260808 20:56:33.620568 21431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:33.620316 21429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:33.621042 21420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:33.621279 21420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:33.622521 21420 hybrid_clock.cc:648] HybridClock initialized: now 1786222593622501 us; error 34 us; skew 500 ppm
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:33.626039 21420 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:33.626590 21420 webserver.cc:460] Webserver started at http://127.15.232.131:46801/ using document root <none> and password file <none>
I20260808 20:56:33.626860 21420 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:33.626935 21420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:33.627113 21420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:33.628383 21420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "7a1ae4c69ff049b6a47ee2a06fadc0b3"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "2dedfb100047f65a63774b0e981d1e75"
server_key_iv: "c49cd0b5fbbe6459a2ccb4dce0470740"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.628813 21420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "7a1ae4c69ff049b6a47ee2a06fadc0b3"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "2dedfb100047f65a63774b0e981d1e75"
server_key_iv: "c49cd0b5fbbe6459a2ccb4dce0470740"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.630343 21420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:33.631201 21444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.631377 21420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:33.631521 21420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "7a1ae4c69ff049b6a47ee2a06fadc0b3"
format_stamp: "Formatted at 2026-08-08 20:56:33 on dist-test-slave-c12x"
server_key: "2dedfb100047f65a63774b0e981d1e75"
server_key_iv: "c49cd0b5fbbe6459a2ccb4dce0470740"
server_key_version: "encryptionkey@0"
I20260808 20:56:33.631618 21420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:33.651098 21420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:33.654132 21420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:33.654336 21420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:33.654635 21420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:33.655027 21420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:33.655092 21420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.655135 21420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:33.655161 21420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:33.662047 21420 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:34163
I20260808 20:56:33.662096 21601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:34163 every 8 connection(s)
I20260808 20:56:33.662916 21420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:33.665252 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 21420
I20260808 20:56:33.665369 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:33.665588 16290 external_mini_cluster.cc:1428] Setting key 07c7d13a2a6ddc70495d6124b237345f
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:33.673717 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:33.663864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:55881 (local address 127.15.232.190:36583)
0808 20:56:33.663947 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:33.663949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:33.664260 (+ 311us) server_negotiation.cc:407] Connection header received
0808 20:56:33.664825 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:33.664826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:33.664837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:33.664899 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:33.666122 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.666943 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.668073 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.668263 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.668329 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:33.670807 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:33.670808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:33.670808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:33.670817 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:33.672409 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.672835 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.672836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.672837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.672895 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.673153 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.673154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.673154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.673266 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:33.673319 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:33.673568 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 20:56:33.673634 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 20:56:33.674024 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:33.663805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36583 (local address 127.15.232.131:55881)
0808 20:56:33.664064 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:33.664146 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:33.664703 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:33.664983 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:33.664986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:33.665712 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:33.665998 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:33.666004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.667026 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.667026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:33.667979 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:33.667982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.668345 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.668346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:33.668457 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:33.669069 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:33.669078 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:33.670704 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:33.672514 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:33.672520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:33.672523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:33.672765 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:33.672960 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:33.672961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:33.672961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:33.673093 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:33.673324 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:33.673329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:33.673453 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 20:56:33.673538 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 20:56:33.673677 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":110,"threads_started":1}
I20260808 20:56:33.674741 21602 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36583
I20260808 20:56:33.674902 21602 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:33.675124 21602 heartbeater.cc:511] Master 127.15.232.190:36583 requested a full tablet report, sending...
I20260808 20:56:33.675697 20964 ts_manager.cc:194] Registered new tserver with Master: 7a1ae4c69ff049b6a47ee2a06fadc0b3 (127.15.232.131:34163)
I20260808 20:56:33.676208 20964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:55881
I20260808 20:56:33.677987 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222593, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[20900](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222593, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:33.706523 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:33.697958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.15.232.190:36583)
0808 20:56:33.698075 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:33.698077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:33.698183 (+ 106us) server_negotiation.cc:407] Connection header received
0808 20:56:33.698289 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:33.698290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:33.698301 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:33.698379 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:33.699002 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.699580 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.700351 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:33.700555 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:33.700642 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:33.703607 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:33.703609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:33.703609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:33.703618 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:33.705183 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.705594 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.705595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.705596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.705667 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:33.705881 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:33.705882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:33.705882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:33.706066 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:33.706182 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:33.706317 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 20:56:33.706411 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 20:56:33.706883 20964 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57762
I20260808 20:56:33.707819 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 21056
I20260808 20:56:33.720789 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 21248
I20260808 20:56:33.732568 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 21420
I20260808 20:56:33.739781 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 20917
2026-08-08T20:56:33Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2139 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[21621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[21621](info): set up 2 sockets
Aug 08 20:56:33 dist-test-slave-c12x krb5kdc[21621](info): commencing operation
krb5kdc: starting...
W20260808 20:56:34.994313 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:35.131016 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36247
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:36247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:35.222064 21641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:35.222303 21641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:35.222328 21641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:35.223636 21641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:35.223682 21641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:35.223696 21641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:35.223711 21641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:35.223722 21641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:35.225327 21641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:36247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:35.225597 21641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:35.225853 21641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:35.231242 21649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:35.231376 21651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:35.231653 21641 server_base.cc:1034] running on GCE node
W20260808 20:56:35.231724 21647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:35.231999 21641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:35.232272 21641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:35.233425 21641 hybrid_clock.cc:648] HybridClock initialized: now 1786222595233405 us; error 42 us; skew 500 ppm
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:35.236187 21641 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:35.236722 21641 webserver.cc:460] Webserver started at http://127.15.232.190:33351/ using document root <none> and password file <none>
I20260808 20:56:35.237010 21641 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:35.237094 21641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:35.237250 21641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:35.238394 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "fd69db2022fd4bba9c7e3878e671b6a8"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "077c7d8d61fa7d5543972446c03bb803"
server_key_iv: "d5b95a49519d05811dffdd17e1749e3d"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.238811 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "fd69db2022fd4bba9c7e3878e671b6a8"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "077c7d8d61fa7d5543972446c03bb803"
server_key_iv: "d5b95a49519d05811dffdd17e1749e3d"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.240450 21641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:35.241518 21662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.241896 21641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:35.242025 21641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "fd69db2022fd4bba9c7e3878e671b6a8"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "077c7d8d61fa7d5543972446c03bb803"
server_key_iv: "d5b95a49519d05811dffdd17e1749e3d"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.242129 21641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:35.257901 21641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:35.260568 21641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:35.260802 21641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:35.265900 21641 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:36247
I20260808 20:56:35.265964 21739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:36247 every 8 connection(s)
I20260808 20:56:35.266531 21641 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:35.267302 21740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:35.268189 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 21641
I20260808 20:56:35.268301 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:35.268482 16290 external_mini_cluster.cc:1428] Setting key 2d5657a74bd0577f69bd0e6cea119229
I20260808 20:56:35.270958 21740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8: Bootstrap starting.
I20260808 20:56:35.271734 21740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:35.272033 21740 log.cc:824] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:35.272778 21740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8: No bootstrap required, opened a new log
I20260808 20:56:35.275074 21740 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd69db2022fd4bba9c7e3878e671b6a8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36247 } }
I20260808 20:56:35.275235 21740 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:35.275261 21740 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd69db2022fd4bba9c7e3878e671b6a8, State: Initialized, Role: FOLLOWER
I20260808 20:56:35.275489 21740 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd69db2022fd4bba9c7e3878e671b6a8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36247 } }
I20260808 20:56:35.275604 21740 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:35.275640 21740 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:35.275691 21740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:35.276571 21740 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd69db2022fd4bba9c7e3878e671b6a8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36247 } }
I20260808 20:56:35.276710 21740 leader_election.cc:302] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd69db2022fd4bba9c7e3878e671b6a8; no voters:
I20260808 20:56:35.276935 21740 leader_election.cc:288] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:35.276993 21748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:35.277170 21748 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [term 1 LEADER]: Becoming Leader. State: Replica: fd69db2022fd4bba9c7e3878e671b6a8, State: Running, Role: LEADER
I20260808 20:56:35.277242 21740 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:35.277345 21748 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd69db2022fd4bba9c7e3878e671b6a8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36247 } }
I20260808 20:56:35.277768 21749 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd69db2022fd4bba9c7e3878e671b6a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd69db2022fd4bba9c7e3878e671b6a8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36247 } } }
I20260808 20:56:35.277853 21749 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:35.278019 21751 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd69db2022fd4bba9c7e3878e671b6a8. Latest consensus state: current_term: 1 leader_uuid: "fd69db2022fd4bba9c7e3878e671b6a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd69db2022fd4bba9c7e3878e671b6a8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36247 } } }
I20260808 20:56:35.278102 21751 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:35.278358 21758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:35.278949 21758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:35.280985 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:35.269141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33812 (local address 127.15.232.190:36247)
0808 20:56:35.269352 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:35.269360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:35.269389 (+ 29us) server_negotiation.cc:407] Connection header received
0808 20:56:35.269897 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:35.269905 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:35.269959 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:35.270087 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:35.270956 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.272207 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.273158 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.273724 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.273874 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:35.276748 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:35.276752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:35.276753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:35.276770 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:35.279091 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.279661 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.279683 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.279684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.279816 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.280046 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.280046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.280047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.280350 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:35.280546 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:35.280656 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 20:56:35.280805 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
I20260808 20:56:35.283958 21758 catalog_manager.cc:1347] Generated new cluster ID: fbc53835813346759005b1e79d6e8732
I20260808 20:56:35.284050 21758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:35.297267 21758 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:35.298030 21758 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:35.304169 21758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd69db2022fd4bba9c7e3878e671b6a8: Generated new TSK 0
I20260808 20:56:35.304405 21758 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:35.335321 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36247
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:35.426610 21774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:35.426828 21774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:35.426852 21774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:35.427980 21774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:35.428018 21774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:35.428043 21774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:35.429546 21774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:35.429742 21774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:35.429921 21774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:35.435359 21780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:35.435374 21786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:35.435353 21781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:35.435418 21774 server_base.cc:1034] running on GCE node
I20260808 20:56:35.435937 21774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:35.436213 21774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:35.437359 21774 hybrid_clock.cc:648] HybridClock initialized: now 1786222595437341 us; error 27 us; skew 500 ppm
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:35.440126 21774 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:35.440590 21774 webserver.cc:460] Webserver started at http://127.15.232.129:33149/ using document root <none> and password file <none>
I20260808 20:56:35.440826 21774 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:35.440897 21774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:35.441061 21774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:35.442200 21774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "d2f5099664834209936077ae25994cf9"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "56655f6041cbf88d5f39d45ba52105a9"
server_key_iv: "4ea893ff14b6dfb6888eed0fc62cee56"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.442597 21774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "d2f5099664834209936077ae25994cf9"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "56655f6041cbf88d5f39d45ba52105a9"
server_key_iv: "4ea893ff14b6dfb6888eed0fc62cee56"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.444144 21774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:35.444945 21795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.445211 21774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:35.445325 21774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "d2f5099664834209936077ae25994cf9"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "56655f6041cbf88d5f39d45ba52105a9"
server_key_iv: "4ea893ff14b6dfb6888eed0fc62cee56"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.445432 21774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:35.460592 21774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:35.463052 21774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:35.463243 21774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:35.463560 21774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:35.463907 21774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:35.463965 21774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.464002 21774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:35.464030 21774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.470163 21774 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:36127
I20260808 20:56:35.470264 21971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:36127 every 8 connection(s)
I20260808 20:56:35.470599 21774 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:35.471673 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 21774
I20260808 20:56:35.471799 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:35.471971 16290 external_mini_cluster.cc:1428] Setting key 7c4f754a6be1d2a77513fe718f0b2f83
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
I20260808 20:56:35.482115 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:35.471507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:53563 (local address 127.15.232.190:36247)
0808 20:56:35.471588 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:35.471590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:35.471740 (+ 150us) server_negotiation.cc:407] Connection header received
0808 20:56:35.472969 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:35.472970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:35.472983 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:35.473025 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:35.473688 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.474203 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.475807 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.476021 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.476088 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:35.479110 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:35.479112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:35.479113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:35.479121 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:35.480627 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.481069 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.481070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.481071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.481133 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.481404 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.481405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.481405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.481529 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:35.481608 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:35.481926 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 20:56:35.482020 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 20:56:35.482303 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:35.471413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36247 (local address 127.15.232.129:53563)
0808 20:56:35.471608 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:35.471654 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:35.472865 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:35.473121 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:35.473124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:35.473390 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:35.473587 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:35.473591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.474306 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.474307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:35.475691 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:35.475695 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.476127 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.476128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:35.476285 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:35.476899 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:35.476910 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:35.479007 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:35.480735 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:35.480743 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:35.480745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:35.480984 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:35.481202 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:35.481203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:35.481204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:35.481323 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:35.481600 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:35.481607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:35.481774 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 20:56:35.481890 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 20:56:35.482062 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
I20260808 20:56:35.482904 21972 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36247
I20260808 20:56:35.483026 21972 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:35.483237 21972 heartbeater.cc:511] Master 127.15.232.190:36247 requested a full tablet report, sending...
I20260808 20:56:35.483937 21685 ts_manager.cc:194] Registered new tserver with Master: d2f5099664834209936077ae25994cf9 (127.15.232.129:36127)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
I20260808 20:56:35.484689 21685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:53563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:35.500982 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36247
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:35.587968 21978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:35.588210 21978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:35.588238 21978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:35.589447 21978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:35.589485 21978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:35.589512 21978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:35.591080 21978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:35.591298 21978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:35.591512 21978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:35.597301 21990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:35.597311 21987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:35.597591 21978 server_base.cc:1034] running on GCE node
W20260808 20:56:35.597316 21985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:35.597890 21978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:35.598110 21978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:35.599267 21978 hybrid_clock.cc:648] HybridClock initialized: now 1786222595599251 us; error 29 us; skew 500 ppm
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:35.601996 21978 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:35.602454 21978 webserver.cc:460] Webserver started at http://127.15.232.130:35017/ using document root <none> and password file <none>
I20260808 20:56:35.602698 21978 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:35.602768 21978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:35.602922 21978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:35.604097 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "d01ac5060c754d01ae5c29bfb69917f1"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "94bd774d5f010b905f414a213fbe11d0"
server_key_iv: "4720613a83662a7024fed1085d30208a"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.604486 21978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "d01ac5060c754d01ae5c29bfb69917f1"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "94bd774d5f010b905f414a213fbe11d0"
server_key_iv: "4720613a83662a7024fed1085d30208a"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.605912 21978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 20:56:35.606715 21999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.606910 21978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:35.607023 21978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "d01ac5060c754d01ae5c29bfb69917f1"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "94bd774d5f010b905f414a213fbe11d0"
server_key_iv: "4720613a83662a7024fed1085d30208a"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.607110 21978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:35.621168 21978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:35.624418 21978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:35.624595 21978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:35.624873 21978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:35.625311 21978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:35.625367 21978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.625404 21978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:35.625427 21978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.632326 21978 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:45601
I20260808 20:56:35.632421 22165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:45601 every 8 connection(s)
I20260808 20:56:35.632781 21978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:35.637272 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 21978
I20260808 20:56:35.637387 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:35.637569 16290 external_mini_cluster.cc:1428] Setting key be975d67752b21ba756b600b15943bfa
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:35.643641 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:35.633595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:49723 (local address 127.15.232.190:36247)
0808 20:56:35.633684 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:35.633687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:35.633877 (+ 190us) server_negotiation.cc:407] Connection header received
0808 20:56:35.634880 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:35.634881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:35.634891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:35.634946 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:35.635639 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.636147 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.637447 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.637737 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.637815 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:35.640438 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:35.640440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:35.640441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:35.640449 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:35.641959 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.642502 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.642504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.642504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.642566 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.642891 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.642891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.642892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.643022 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:35.643076 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:35.643479 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 20:56:35.643573 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 20:56:35.643955 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:35.633542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36247 (local address 127.15.232.130:49723)
0808 20:56:35.633749 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:35.633787 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:35.634772 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:35.635003 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:35.635006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:35.635274 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:35.635559 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:35.635564 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.636246 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.636247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:35.637309 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:35.637313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.637842 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.637844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:35.638020 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:35.638634 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:35.638643 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:35.640324 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:35.642054 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:35.642063 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:35.642065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:35.642416 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:35.642646 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:35.642647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:35.642647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:35.642818 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:35.643074 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:35.643082 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:35.643279 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 20:56:35.643411 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 20:56:35.643633 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":112,"threads_started":1}
I20260808 20:56:35.644584 22166 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36247
I20260808 20:56:35.644706 22166 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:35.644904 22166 heartbeater.cc:511] Master 127.15.232.190:36247 requested a full tablet report, sending...
I20260808 20:56:35.645363 21685 ts_manager.cc:194] Registered new tserver with Master: d01ac5060c754d01ae5c29bfb69917f1 (127.15.232.130:45601)
I20260808 20:56:35.645825 21685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:49723
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:35.667508 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36247
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:35.753377 22174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:35.753679 22174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:35.753733 22174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:35.754921 22174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:35.755009 22174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:35.755060 22174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:35.756927 22174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:35.757228 22174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:35.757491 22174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:35.763094 22181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:35.763137 22183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:35.763103 22180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:35.763141 22174 server_base.cc:1034] running on GCE node
I20260808 20:56:35.763614 22174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:35.763890 22174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:35.765064 22174 hybrid_clock.cc:648] HybridClock initialized: now 1786222595765043 us; error 39 us; skew 500 ppm
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:35.767992 22174 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:35.768513 22174 webserver.cc:460] Webserver started at http://127.15.232.131:33333/ using document root <none> and password file <none>
I20260808 20:56:35.768764 22174 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:35.768834 22174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:35.769008 22174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:35.770179 22174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "b83bbcbe492744e1aea6d3717d8a3c85"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "e53d7dde164302ff354105fd8860be44"
server_key_iv: "dfcbb6d5f7fae8f25c79b1671ad633e4"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.770571 22174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "b83bbcbe492744e1aea6d3717d8a3c85"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "e53d7dde164302ff354105fd8860be44"
server_key_iv: "dfcbb6d5f7fae8f25c79b1671ad633e4"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.772167 22174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:35.773180 22190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.773438 22174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:35.773569 22174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "b83bbcbe492744e1aea6d3717d8a3c85"
format_stamp: "Formatted at 2026-08-08 20:56:35 on dist-test-slave-c12x"
server_key: "e53d7dde164302ff354105fd8860be44"
server_key_iv: "dfcbb6d5f7fae8f25c79b1671ad633e4"
server_key_version: "encryptionkey@0"
I20260808 20:56:35.773664 22174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:35.793792 22174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:35.796240 22174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:35.796423 22174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:35.796686 22174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:35.797012 22174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:35.797068 22174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.797111 22174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:35.797142 22174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:35.803205 22174 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:34155
I20260808 20:56:35.803247 22349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:34155 every 8 connection(s)
I20260808 20:56:35.803655 22174 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:35.803975 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 22174
I20260808 20:56:35.804104 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:35.804322 16290 external_mini_cluster.cc:1428] Setting key cf1757f43c6928d51f6b2fd7a24a946e
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:35.813795 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:35.804503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:54521 (local address 127.15.232.190:36247)
0808 20:56:35.804554 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:35.804556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:35.804852 (+ 296us) server_negotiation.cc:407] Connection header received
0808 20:56:35.806029 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:35.806030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:35.806041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:35.806103 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:35.806727 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.807232 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.808285 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.808478 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.808535 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:35.810883 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:35.810884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:35.810885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:35.810892 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:35.812457 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.812877 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.812879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.812879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.812939 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.813220 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.813221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.813222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.813333 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:35.813388 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:35.813661 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 20:56:35.813729 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:56:35.814060 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:35.804501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36247 (local address 127.15.232.131:54521)
0808 20:56:35.804686 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:35.804739 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:35.805910 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:35.806163 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:35.806166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:35.806430 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:35.806632 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:35.806636 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.807315 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.807316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:35.808201 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:35.808203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.808557 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.808557 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:35.808664 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:35.809165 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:35.809174 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:35.810699 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:35.812540 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:35.812547 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:35.812549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:35.812790 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:35.813032 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:35.813033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:35.813033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:35.813149 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:35.813410 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:35.813415 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:35.813539 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 20:56:35.813635 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 20:56:35.813777 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260808 20:56:35.814756 22351 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36247
I20260808 20:56:35.814852 22351 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:35.815064 22351 heartbeater.cc:511] Master 127.15.232.190:36247 requested a full tablet report, sending...
I20260808 20:56:35.815613 21685 ts_manager.cc:194] Registered new tserver with Master: b83bbcbe492744e1aea6d3717d8a3c85 (127.15.232.131:34155)
I20260808 20:56:35.816172 21685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:54521
I20260808 20:56:35.816269 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:35.841799 22363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:35.833491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38698 (local address 127.15.232.129:36127)
0808 20:56:35.833669 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:35.833678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:35.833695 (+ 17us) server_negotiation.cc:407] Connection header received
0808 20:56:35.833722 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:35.833724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:35.833738 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:35.833824 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:35.834451 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.835214 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.836051 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.836382 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.836483 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:35.838815 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:35.838824 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:35.838824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:35.838831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:35.840679 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.841062 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.841065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.841065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.841148 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.841332 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.841332 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.841333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.841478 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:35.841611 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:35.841651 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 20:56:35.841714 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:35.850718 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:35.843369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.15.232.190:36247)
0808 20:56:35.843423 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:35.843425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:35.843566 (+ 141us) server_negotiation.cc:407] Connection header received
0808 20:56:35.843612 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:35.843613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:35.843622 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:35.843674 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:35.844182 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.844762 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.845449 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:35.845638 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:35.845693 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:35.848060 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:35.848062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:35.848063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:35.848071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:35.849467 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.849816 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.849817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.849817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.849883 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:35.850090 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:35.850091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:35.850092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:35.850262 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:35.850372 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:35.850508 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 20:56:35.850610 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 20:56:35.851008 21685 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33818
I20260808 20:56:35.851495 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 21774
I20260808 20:56:35.866420 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 21978
I20260808 20:56:35.881695 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 22174
I20260808 20:56:35.893708 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 21641
2026-08-08T20:56:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2150 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[22370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[22370](info): set up 2 sockets
Aug 08 20:56:35 dist-test-slave-c12x krb5kdc[22370](info): commencing operation
krb5kdc: starting...
W20260808 20:56:37.152504 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:37 dist-test-slave-c12x krb5kdc[22370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:37.286875 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:45239
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:45239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:37.373566 22390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:37.373854 22390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:37.373899 22390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:37.375357 22390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:37.375488 22390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:37.375540 22390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:37.375571 22390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:37.375599 22390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:37.377544 22390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:45239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:45239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:37.377882 22390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:37.378145 22390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:37.383603 22399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:37.383598 22403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:37.383658 22390 server_base.cc:1034] running on GCE node
W20260808 20:56:37.383598 22398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:37.384256 22390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:37.384619 22390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:37.385816 22390 hybrid_clock.cc:648] HybridClock initialized: now 1786222597385813 us; error 58 us; skew 500 ppm
Aug 08 20:56:37 dist-test-slave-c12x krb5kdc[22370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222597, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:37.392333 22390 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:37.392765 22390 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:37.392822 22390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:37.392930 22390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:37.394065 22390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "ef9597f301ce40dea576ddf9a7724228"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "d866448aa9233cb61cf7bf089885a334"
server_key_iv: "124b01a88f61015b22ab885a83c227b6"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.394476 22390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "ef9597f301ce40dea576ddf9a7724228"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "d866448aa9233cb61cf7bf089885a334"
server_key_iv: "124b01a88f61015b22ab885a83c227b6"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.396035 22390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:37.397018 22411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:37.397289 22390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:37.397377 22390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "ef9597f301ce40dea576ddf9a7724228"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "d866448aa9233cb61cf7bf089885a334"
server_key_iv: "124b01a88f61015b22ab885a83c227b6"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.397501 22390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:37.405718 22390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:37.408511 22390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:37.408700 22390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:37.415508 22390 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:45239
I20260808 20:56:37.415556 22481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:45239 every 8 connection(s)
I20260808 20:56:37.416008 22390 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:37.417001 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:37.419351 22482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228: Bootstrap starting.
I20260808 20:56:37.420033 22482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:37.420295 22482 log.cc:824] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:37.421036 22482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228: No bootstrap required, opened a new log
I20260808 20:56:37.422653 22482 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9597f301ce40dea576ddf9a7724228" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 45239 } }
I20260808 20:56:37.422802 22482 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:37.422827 22482 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9597f301ce40dea576ddf9a7724228, State: Initialized, Role: FOLLOWER
I20260808 20:56:37.422991 22482 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9597f301ce40dea576ddf9a7724228" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 45239 } }
I20260808 20:56:37.423072 22482 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:37.423096 22482 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:37.423125 22482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:37.423640 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 22390
I20260808 20:56:37.423774 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:37.424014 16290 external_mini_cluster.cc:1428] Setting key f24c6ea08309169c36dd9522b2af891e
I20260808 20:56:37.424289 22482 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9597f301ce40dea576ddf9a7724228" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 45239 } }
I20260808 20:56:37.424417 22482 leader_election.cc:302] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef9597f301ce40dea576ddf9a7724228; no voters:
I20260808 20:56:37.424659 22482 leader_election.cc:288] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:37.424722 22488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:37.424921 22488 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [term 1 LEADER]: Becoming Leader. State: Replica: ef9597f301ce40dea576ddf9a7724228, State: Running, Role: LEADER
I20260808 20:56:37.425083 22488 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9597f301ce40dea576ddf9a7724228" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 45239 } }
I20260808 20:56:37.425623 22491 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef9597f301ce40dea576ddf9a7724228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9597f301ce40dea576ddf9a7724228" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 45239 } } }
I20260808 20:56:37.425657 22490 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef9597f301ce40dea576ddf9a7724228. Latest consensus state: current_term: 1 leader_uuid: "ef9597f301ce40dea576ddf9a7724228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9597f301ce40dea576ddf9a7724228" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 45239 } } }
I20260808 20:56:37.425810 22491 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:37.425844 22490 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:37.425091 22482 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:37.426348 22493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:37.427057 22493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:37.428859 22493 catalog_manager.cc:1347] Generated new cluster ID: dbb4e3ea56ce476bb14bcda9d42090c4
I20260808 20:56:37.428920 22493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 20:56:37 dist-test-slave-c12x krb5kdc[22370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:37.437405 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:37.424558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55876 (local address 127.15.232.190:45239)
0808 20:56:37.424786 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:37.424797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:37.424842 (+ 45us) server_negotiation.cc:407] Connection header received
0808 20:56:37.424983 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:37.424992 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:37.425078 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:37.425243 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:37.426014 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.427577 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.428659 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.429266 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.429368 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:37.432500 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:37.432505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:37.432506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:37.432520 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:37.435575 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:37.436035 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:37.436038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:37.436039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:37.436164 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:37.436381 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:37.436381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:37.436382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:37.436742 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:37.436920 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:37.436997 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 20:56:37.437162 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260808 20:56:37.443303 22493 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:37.443912 22493 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:37.449838 22493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228: Generated new TSK 0
I20260808 20:56:37.450100 22493 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:37.487064 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:45239
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:37.577455 22512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:37.577682 22512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:37.577709 22512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:37.578826 22512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:37.578861 22512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:37.578888 22512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:37.580492 22512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:45239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:37.580694 22512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:37.580883 22512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:37.586135 22521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:37.586269 22518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:37.586136 22517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:37.586674 22512 server_base.cc:1034] running on GCE node
I20260808 20:56:37.586956 22512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:37.587212 22512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:37.588388 22512 hybrid_clock.cc:648] HybridClock initialized: now 1786222597588373 us; error 27 us; skew 500 ppm
Aug 08 20:56:37 dist-test-slave-c12x krb5kdc[22370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222597, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:37.591096 22512 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:37.591548 22512 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:37.591601 22512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:37.591701 22512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:37.592754 22512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "71c79bfbc68b419f9c097b6adb8e457b"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "76939ba601b8db9347774e17450caf35"
server_key_iv: "e005ec160e7baee7c2ae7fd1e5068d05"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.593079 22512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "71c79bfbc68b419f9c097b6adb8e457b"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "76939ba601b8db9347774e17450caf35"
server_key_iv: "e005ec160e7baee7c2ae7fd1e5068d05"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.594412 22512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:37.595145 22532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:37.595337 22512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 20:56:37.595480 22512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "71c79bfbc68b419f9c097b6adb8e457b"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "76939ba601b8db9347774e17450caf35"
server_key_iv: "e005ec160e7baee7c2ae7fd1e5068d05"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.595548 22512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:37.637446 22512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:37.639611 22512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:37.639736 22512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:37.640027 22512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:37.640393 22512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:37.640453 22512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:37.640497 22512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:37.640525 22512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:37.646461 22512 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:35487
I20260808 20:56:37.646502 22685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:35487 every 8 connection(s)
I20260808 20:56:37.646782 22512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
Aug 08 20:56:37 dist-test-slave-c12x krb5kdc[22370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222597, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:37.653899 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 22512
I20260808 20:56:37.654057 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:37.654263 16290 external_mini_cluster.cc:1428] Setting key 5cb9b18c2b92f1b96d5d643d6f26851f
I20260808 20:56:37.657008 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:37.647521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:43915 (local address 127.15.232.190:45239)
0808 20:56:37.647581 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:37.647585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:37.647838 (+ 253us) server_negotiation.cc:407] Connection header received
0808 20:56:37.648291 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:37.648292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:37.648304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:37.648370 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:37.649345 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.649875 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.650883 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.651099 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.651160 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:37.653587 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:37.653588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:37.653589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:37.653599 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:37.655200 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:37.655816 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:37.655817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:37.655818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:37.655876 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:37.656184 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:37.656185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:37.656185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:37.656303 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:37.656354 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:37.656766 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 20:56:37.656879 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 20:56:37.657173 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:37.647511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:45239 (local address 127.15.232.129:43915)
0808 20:56:37.647677 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:37.647720 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:37.648192 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:37.648452 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:37.648456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:37.648718 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:37.649237 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:37.649242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.649972 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.649973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:37.650774 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:37.650776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.651199 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.651201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:37.651313 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:37.651879 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:37.651888 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:37.653478 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:37.655283 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:37.655292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:37.655294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:37.655717 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:37.655953 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:37.655954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:37.655954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:37.656109 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:37.656367 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:37.656375 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:37.656563 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 20:56:37.656675 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 20:56:37.656878 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260808 20:56:37.657879 22686 heartbeater.cc:347] Connected to a master server at 127.15.232.190:45239
I20260808 20:56:37.657989 22686 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:37.658147 22686 heartbeater.cc:511] Master 127.15.232.190:45239 requested a full tablet report, sending...
I20260808 20:56:37.658671 22433 ts_manager.cc:194] Registered new tserver with Master: 71c79bfbc68b419f9c097b6adb8e457b (127.15.232.129:35487)
I20260808 20:56:37.659328 22433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:43915
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:37.686892 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:45239
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:37.772118 22692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:37.772374 22692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:37.772419 22692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:37.773612 22692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:37.773655 22692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:37.773684 22692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:37.775193 22692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:45239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:37.775391 22692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:37.775671 22692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:37.780807 22708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:37.780831 22701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:37.781458 22704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:37.781535 22692 server_base.cc:1034] running on GCE node
I20260808 20:56:37.781740 22692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:37.781960 22692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:37.783113 22692 hybrid_clock.cc:648] HybridClock initialized: now 1786222597783061 us; error 66 us; skew 500 ppm
Aug 08 20:56:37 dist-test-slave-c12x krb5kdc[22370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222597, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:37.785969 22692 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:37.786378 22692 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:37.786463 22692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:37.786629 22692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:37.787894 22692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "117e39e7c2754affaa0402aa336caf90"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "873674b9864b981cbf9c4d2c6d0d0f15"
server_key_iv: "f9b65126ff238caeb04f10501e407665"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.788287 22692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "117e39e7c2754affaa0402aa336caf90"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "873674b9864b981cbf9c4d2c6d0d0f15"
server_key_iv: "f9b65126ff238caeb04f10501e407665"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.789912 22692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:37.790732 22719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:37.790932 22692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:37.791046 22692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "117e39e7c2754affaa0402aa336caf90"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "873674b9864b981cbf9c4d2c6d0d0f15"
server_key_iv: "f9b65126ff238caeb04f10501e407665"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.791126 22692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:37.838426 22692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:37.841037 22692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:37.841224 22692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:37.841497 22692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:37.841868 22692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:37.841924 22692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:37.841969 22692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:37.842021 22692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:37.849254 22692 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:46041
I20260808 20:56:37.849289 22884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:46041 every 8 connection(s)
I20260808 20:56:37.849594 22692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:37.853859 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 22692
I20260808 20:56:37.853997 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:37.854185 16290 external_mini_cluster.cc:1428] Setting key ad1c5e93ac61b23695b667064727253f
Aug 08 20:56:37 dist-test-slave-c12x krb5kdc[22370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222597, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:37.860602 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:37.850400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:59077 (local address 127.15.232.190:45239)
0808 20:56:37.850448 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:37.850450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:37.850696 (+ 246us) server_negotiation.cc:407] Connection header received
0808 20:56:37.851194 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:37.851195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:37.851206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:37.851263 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:37.852303 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.852859 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.854022 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.854326 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.854465 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:37.857271 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:37.857272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:37.857273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:37.857280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:37.859036 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:37.859465 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:37.859466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:37.859467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:37.859543 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:37.859819 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:37.859820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:37.859820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:37.859990 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:37.860073 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:37.860387 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 20:56:37.860499 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 20:56:37.860647 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:37.850386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:45239 (local address 127.15.232.130:59077)
0808 20:56:37.850550 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:37.850586 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:37.851089 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:37.851319 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:37.851322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:37.851677 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:37.852193 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:37.852199 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.852973 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.852975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:37.853888 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:37.853891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:37.854492 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:37.854493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:37.854647 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:37.855280 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:37.855289 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:37.857164 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:37.859131 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:37.859138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:37.859140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:37.859364 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:37.859619 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:37.859620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:37.859620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:37.859741 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:37.860081 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:37.860087 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:37.860234 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 20:56:37.860335 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 20:56:37.860467 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260808 20:56:37.861243 22885 heartbeater.cc:347] Connected to a master server at 127.15.232.190:45239
I20260808 20:56:37.861366 22885 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:37.861567 22885 heartbeater.cc:511] Master 127.15.232.190:45239 requested a full tablet report, sending...
I20260808 20:56:37.862030 22433 ts_manager.cc:194] Registered new tserver with Master: 117e39e7c2754affaa0402aa336caf90 (127.15.232.130:46041)
I20260808 20:56:37.862435 22433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:59077
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:37.884907 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:45239
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:37.973037 22891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:37.973263 22891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:37.973289 22891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:37.974506 22891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:37.974561 22891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:37.974591 22891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:37.976295 22891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:45239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:37.976583 22891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:37.976833 22891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:37.982188 22896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:37.982189 22899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:37.982296 22897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:37.982347 22891 server_base.cc:1034] running on GCE node
I20260808 20:56:37.982704 22891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:37.982995 22891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:37.984140 22891 hybrid_clock.cc:648] HybridClock initialized: now 1786222597984115 us; error 40 us; skew 500 ppm
Aug 08 20:56:37 dist-test-slave-c12x krb5kdc[22370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222597, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:37.987010 22891 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:37.987497 22891 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:37.987573 22891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:37.987672 22891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:37.988972 22891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "19b3152b44c34698bda5d4bff09c1c1c"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "8d687084a3c2dc43f35fb238cfb5f178"
server_key_iv: "d717861e17e8a0942e6531edd630a331"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.989326 22891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "19b3152b44c34698bda5d4bff09c1c1c"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "8d687084a3c2dc43f35fb238cfb5f178"
server_key_iv: "d717861e17e8a0942e6531edd630a331"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.990664 22891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:37.991508 22905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:37.991694 22891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:37.991793 22891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "19b3152b44c34698bda5d4bff09c1c1c"
format_stamp: "Formatted at 2026-08-08 20:56:37 on dist-test-slave-c12x"
server_key: "8d687084a3c2dc43f35fb238cfb5f178"
server_key_iv: "d717861e17e8a0942e6531edd630a331"
server_key_version: "encryptionkey@0"
I20260808 20:56:37.991858 22891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:38.015826 22891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:38.018564 22891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:38.018687 22891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:38.018890 22891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:38.019244 22891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:38.019280 22891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.019330 22891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:38.019362 22891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.025259 22891 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:45061
I20260808 20:56:38.025292 23080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:45061 every 8 connection(s)
I20260808 20:56:38.025611 22891 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:38.031630 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 22891
I20260808 20:56:38.031745 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:38.031915 16290 external_mini_cluster.cc:1428] Setting key a7425aae89e8f669d9759812e59fdb52
Aug 08 20:56:38 dist-test-slave-c12x krb5kdc[22370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222597, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:38.035648 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.026333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:53631 (local address 127.15.232.190:45239)
0808 20:56:38.026408 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.026410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.026652 (+ 242us) server_negotiation.cc:407] Connection header received
0808 20:56:38.027214 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.027216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.027226 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.027271 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.028276 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.028772 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.029776 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.029977 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.030042 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.032591 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.032593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.032594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:38.032602 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.034123 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:38.034620 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:38.034621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:38.034621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:38.034683 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:38.034948 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:38.034950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:38.034950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:38.035075 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.035154 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.035458 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.035553 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 20:56:38.035889 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.026319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:45239 (local address 127.15.232.131:53631)
0808 20:56:38.026519 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.026562 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.027089 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.027330 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.027334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.027676 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:38.028192 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.028197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.028853 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.028854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.029679 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.029681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.030089 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.030090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.030203 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.030702 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:38.030711 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:38.032498 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:38.034229 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:38.034237 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:38.034239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:38.034537 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:38.034761 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:38.034762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:38.034763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:38.034877 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:38.035145 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:38.035152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:38.035300 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.035412 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.035613 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260808 20:56:38.036412 23082 heartbeater.cc:347] Connected to a master server at 127.15.232.190:45239
I20260808 20:56:38.036545 23082 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:38.036710 23082 heartbeater.cc:511] Master 127.15.232.190:45239 requested a full tablet report, sending...
I20260808 20:56:38.037111 22433 ts_manager.cc:194] Registered new tserver with Master: 19b3152b44c34698bda5d4bff09c1c1c (127.15.232.131:45061)
I20260808 20:56:38.037492 22433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:53631
I20260808 20:56:38.043766 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:56:38.051252 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.045159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55886 (local address 127.15.232.190:45239)
0808 20:56:38.045241 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.045244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.045317 (+ 73us) server_negotiation.cc:407] Connection header received
0808 20:56:38.045398 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.045399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.045410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.045467 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.046050 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.046556 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.047290 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.047559 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.047634 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.048783 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.048784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.048785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:38.048793 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.050032 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:38.050379 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:38.050380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:38.050380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:38.050437 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:38.050669 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:38.050669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:38.050670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:38.050832 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.050945 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.051064 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.051148 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 20:56:38.053125 22433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:38.054802 22433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:38.062000 23095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.057848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35487 (local address 127.0.0.1:47632)
0808 20:56:38.058057 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.058132 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.058180 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.058550 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.058551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.058554 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:38.058589 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.058590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.059920 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.059921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.060991 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.060993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.061680 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.061681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.061762 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.061775 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.061844 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.061902 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":80,"threads_started":1}
I20260808 20:56:38.062080 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.057850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.15.232.130:46041)
0808 20:56:38.058303 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.058308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.058380 (+ 72us) server_negotiation.cc:407] Connection header received
0808 20:56:38.058427 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.058457 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.058471 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.058710 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:38.058792 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.059860 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.060982 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.061739 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.061816 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.061870 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.061944 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.061986 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":90,"threads_started":1}
I20260808 20:56:38.062451 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.057740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.15.232.131:45061)
0808 20:56:38.058123 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.058129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.058141 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:56:38.058186 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.058188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.058206 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.058290 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:38.058503 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.059406 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.061048 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.061739 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.061819 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.061870 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.062326 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.062371 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":113,"threads_started":1}
I20260808 20:56:38.062601 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.058093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.15.232.129:35487)
0808 20:56:38.058391 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.058395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.058404 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:38.058439 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.058441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.058456 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.058551 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:38.058641 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.059804 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.061084 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.061585 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.061684 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.061744 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.062468 (+ 724us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.062516 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":83,"threads_started":1}
I20260808 20:56:38.062129 23100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.057613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:46041 (local address 127.0.0.1:41966)
0808 20:56:38.057984 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.057995 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.058053 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.058577 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.058578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.058581 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:38.058631 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.058633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.059981 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.059982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.060876 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.060879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.061825 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.061826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.061879 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.061882 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.061931 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.061961 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":41,"thread_start_us":85,"threads_started":1}
I20260808 20:56:38.062263 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.057612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:45061 (local address 127.0.0.1:53344)
0808 20:56:38.057817 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.057857 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.057949 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.058293 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.058296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.058298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:38.058401 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.058403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.059559 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.059560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.060963 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.060966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.062039 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.062039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.062096 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.062098 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.062153 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.062203 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
I20260808 20:56:38.063520 22790 tablet_service.cc:1467] Processing CreateTablet for tablet 386c74c3876d42e9b202318b32724209 (DEFAULT_TABLE table=test-table [id=efc93bb3c2934ccaa9f60641b23993ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:38.063519 22594 tablet_service.cc:1467] Processing CreateTablet for tablet 386c74c3876d42e9b202318b32724209 (DEFAULT_TABLE table=test-table [id=efc93bb3c2934ccaa9f60641b23993ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:38.063920 22790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386c74c3876d42e9b202318b32724209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:38.063923 22594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386c74c3876d42e9b202318b32724209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:38.063848 22976 tablet_service.cc:1467] Processing CreateTablet for tablet 386c74c3876d42e9b202318b32724209 (DEFAULT_TABLE table=test-table [id=efc93bb3c2934ccaa9f60641b23993ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:38.064148 22976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 386c74c3876d42e9b202318b32724209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:38.067340 23109 tablet_bootstrap.cc:492] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b: Bootstrap starting.
I20260808 20:56:38.067499 23106 tablet_bootstrap.cc:492] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: Bootstrap starting.
I20260808 20:56:38.068174 23106 tablet_bootstrap.cc:654] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:38.068174 23109 tablet_bootstrap.cc:654] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:38.068852 23109 log.cc:824] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:38.069160 23108 tablet_bootstrap.cc:492] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: Bootstrap starting.
I20260808 20:56:38.069649 23108 tablet_bootstrap.cc:654] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:38.070077 23108 log.cc:824] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:38.070492 23109 tablet_bootstrap.cc:492] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b: No bootstrap required, opened a new log
I20260808 20:56:38.070547 23109 ts_tablet_manager.cc:1397] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 20:56:38.071244 23106 log.cc:824] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:38.071866 23108 tablet_bootstrap.cc:492] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: No bootstrap required, opened a new log
I20260808 20:56:38.071969 23108 ts_tablet_manager.cc:1397] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 20:56:38.072180 23106 tablet_bootstrap.cc:492] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: No bootstrap required, opened a new log
I20260808 20:56:38.072254 23106 ts_tablet_manager.cc:1397] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260808 20:56:38.072577 23109 raft_consensus.cc:348] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } }
I20260808 20:56:38.072769 23109 raft_consensus.cc:374] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:38.072806 23109 raft_consensus.cc:729] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71c79bfbc68b419f9c097b6adb8e457b, State: Initialized, Role: FOLLOWER
I20260808 20:56:38.072978 23109 consensus_queue.cc:260] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } }
I20260808 20:56:38.073273 23109 ts_tablet_manager.cc:1428] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 20:56:38.073405 22686 heartbeater.cc:503] Master 127.15.232.190:45239 was elected leader, sending a full tablet report...
I20260808 20:56:38.073815 23108 raft_consensus.cc:348] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } }
I20260808 20:56:38.073987 23108 raft_consensus.cc:374] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:38.074062 23108 raft_consensus.cc:729] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b3152b44c34698bda5d4bff09c1c1c, State: Initialized, Role: FOLLOWER
I20260808 20:56:38.074188 23108 consensus_queue.cc:260] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } }
I20260808 20:56:38.074502 23108 ts_tablet_manager.cc:1428] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 20:56:38.074589 23082 heartbeater.cc:503] Master 127.15.232.190:45239 was elected leader, sending a full tablet report...
I20260808 20:56:38.074695 23106 raft_consensus.cc:348] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } }
I20260808 20:56:38.074865 23106 raft_consensus.cc:374] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:38.074918 23106 raft_consensus.cc:729] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117e39e7c2754affaa0402aa336caf90, State: Initialized, Role: FOLLOWER
I20260808 20:56:38.075039 23106 consensus_queue.cc:260] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } }
I20260808 20:56:38.075341 23106 ts_tablet_manager.cc:1428] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260808 20:56:38.075388 22885 heartbeater.cc:503] Master 127.15.232.190:45239 was elected leader, sending a full tablet report...
W20260808 20:56:38.100390 22887 tablet.cc:2369] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 20:56:38.147568 22687 tablet.cc:2369] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:38.245203 23116 raft_consensus.cc:482] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:38.245424 23116 raft_consensus.cc:504] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } }
I20260808 20:56:38.246009 23116 leader_election.cc:288] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b3152b44c34698bda5d4bff09c1c1c (127.15.232.131:45061), 71c79bfbc68b419f9c097b6adb8e457b (127.15.232.129:35487)
I20260808 20:56:38.249063 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.246258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:45061 (local address 127.15.232.130:37937)
0808 20:56:38.246348 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.246360 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.246446 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.246669 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.246670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.246672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:38.246748 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.246750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.247400 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.247401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.248218 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.248220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.248785 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.248786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.248869 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.248872 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.248937 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.248978 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 20:56:38.249053 23098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.246376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:37937 (local address 127.15.232.131:45061)
0808 20:56:38.246463 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.246465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.246476 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:38.246566 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.246567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.246579 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.246647 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:38.246838 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.247316 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.248317 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.248697 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.248788 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.248797 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.248955 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.248988 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 20:56:38.249279 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.246440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35487 (local address 127.15.232.130:40323)
0808 20:56:38.246631 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.246641 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.246733 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.246905 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.246906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.246908 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:38.246976 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.246978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.247696 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.247697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.248424 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:38.248426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.248965 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.248966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:38.249036 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.249046 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.249136 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.249182 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":135,"threads_started":1}
I20260808 20:56:38.249362 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.246520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:40323 (local address 127.15.232.129:35487)
0808 20:56:38.246576 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.246578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.246727 (+ 149us) server_negotiation.cc:407] Connection header received
0808 20:56:38.246795 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.246796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.246806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.246893 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:38.247061 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.247612 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.248520 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.248880 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.248966 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.248973 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.249245 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.249288 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 20:56:38.249691 22620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c74c3876d42e9b202318b32724209" candidate_uuid: "117e39e7c2754affaa0402aa336caf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71c79bfbc68b419f9c097b6adb8e457b" is_pre_election: true
I20260808 20:56:38.249676 23007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c74c3876d42e9b202318b32724209" candidate_uuid: "117e39e7c2754affaa0402aa336caf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b3152b44c34698bda5d4bff09c1c1c" is_pre_election: true
I20260808 20:56:38.249845 22620 raft_consensus.cc:2454] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117e39e7c2754affaa0402aa336caf90 in term 0.
I20260808 20:56:38.249854 23007 raft_consensus.cc:2454] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117e39e7c2754affaa0402aa336caf90 in term 0.
I20260808 20:56:38.250074 22722 leader_election.cc:302] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117e39e7c2754affaa0402aa336caf90, 71c79bfbc68b419f9c097b6adb8e457b; no voters:
I20260808 20:56:38.250200 23116 raft_consensus.cc:2781] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:38.250257 23116 raft_consensus.cc:482] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:38.250285 23116 raft_consensus.cc:3037] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:38.250996 23116 raft_consensus.cc:504] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } }
I20260808 20:56:38.251140 23116 leader_election.cc:288] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [CANDIDATE]: Term 1 election: Requested vote from peers 19b3152b44c34698bda5d4bff09c1c1c (127.15.232.131:45061), 71c79bfbc68b419f9c097b6adb8e457b (127.15.232.129:35487)
I20260808 20:56:38.251314 23007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c74c3876d42e9b202318b32724209" candidate_uuid: "117e39e7c2754affaa0402aa336caf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b3152b44c34698bda5d4bff09c1c1c"
I20260808 20:56:38.251322 22620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "386c74c3876d42e9b202318b32724209" candidate_uuid: "117e39e7c2754affaa0402aa336caf90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71c79bfbc68b419f9c097b6adb8e457b"
I20260808 20:56:38.251390 23007 raft_consensus.cc:3037] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:38.251413 22620 raft_consensus.cc:3037] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:38.252120 23007 raft_consensus.cc:2454] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117e39e7c2754affaa0402aa336caf90 in term 1.
I20260808 20:56:38.252245 22620 raft_consensus.cc:2454] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117e39e7c2754affaa0402aa336caf90 in term 1.
I20260808 20:56:38.252307 22722 leader_election.cc:302] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117e39e7c2754affaa0402aa336caf90, 19b3152b44c34698bda5d4bff09c1c1c; no voters:
I20260808 20:56:38.252425 23116 raft_consensus.cc:2781] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:38.252635 23116 raft_consensus.cc:686] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 1 LEADER]: Becoming Leader. State: Replica: 117e39e7c2754affaa0402aa336caf90, State: Running, Role: LEADER
I20260808 20:56:38.252756 23116 consensus_queue.cc:237] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } }
I20260808 20:56:38.253707 22433 catalog_manager.cc:5495] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 reported cstate change: term changed from 0 to 1, leader changed from <none> to 117e39e7c2754affaa0402aa336caf90 (127.15.232.130). New cstate: current_term: 1 leader_uuid: "117e39e7c2754affaa0402aa336caf90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117e39e7c2754affaa0402aa336caf90" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46041 } health_report { overall_health: HEALTHY } } }
W20260808 20:56:38.276144 23083 tablet.cc:2369] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:38.290968 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.288468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.15.232.130:46041)
0808 20:56:38.288514 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.288516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.288523 (+ 7us) server_negotiation.cc:407] Connection header received
0808 20:56:38.288558 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.288559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.288577 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.288631 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:38.288796 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.289288 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.290018 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:38.290218 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:38.290275 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:38.290511 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:38.290624 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:38.290789 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.290812 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.290834 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:56:38.293079 23007 raft_consensus.cc:1261] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [term 1 FOLLOWER]: Refusing update from remote peer 117e39e7c2754affaa0402aa336caf90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:38.293108 22620 raft_consensus.cc:1261] T 386c74c3876d42e9b202318b32724209 P 71c79bfbc68b419f9c097b6adb8e457b [term 1 FOLLOWER]: Refusing update from remote peer 117e39e7c2754affaa0402aa336caf90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:38.293470 23118 consensus_queue.cc:1035] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b3152b44c34698bda5d4bff09c1c1c" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 45061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:38.293619 23116 consensus_queue.cc:1035] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71c79bfbc68b419f9c097b6adb8e457b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:38.296150 23121 mvcc.cc:205] Tried to move back new op lower bound from 7316367762605080576 to 7316367762443743232. Current Snapshot: MvccSnapshot[applied={T|T < 7316367762605080576}]
I20260808 20:56:38.297561 23122 mvcc.cc:205] Tried to move back new op lower bound from 7316367762605080576 to 7316367762443743232. Current Snapshot: MvccSnapshot[applied={T|T < 7316367762605080576 or (T in {7316367762605080576})}]
I20260808 20:56:38.299021 22432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:38.299127 22432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:38.300181 22432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef9597f301ce40dea576ddf9a7724228: Sending DeleteTablet for 3 replicas of tablet 386c74c3876d42e9b202318b32724209
I20260808 20:56:38.300570 22976 tablet_service.cc:1514] Processing DeleteTablet for tablet 386c74c3876d42e9b202318b32724209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:38 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:53344
I20260808 20:56:38.300595 22790 tablet_service.cc:1514] Processing DeleteTablet for tablet 386c74c3876d42e9b202318b32724209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:38 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:41966
I20260808 20:56:38.300935 23140 tablet_replica.cc:330] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: stopping tablet replica
I20260808 20:56:38.300992 22594 tablet_service.cc:1514] Processing DeleteTablet for tablet 386c74c3876d42e9b202318b32724209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:38 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:47632
I20260808 20:56:38.301110 23140 raft_consensus.cc:2229] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:38.301185 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 22512
I20260808 20:56:38.304844 23140 raft_consensus.cc:2258] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:38.305086 23139 tablet_replica.cc:330] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: stopping tablet replica
I20260808 20:56:38.305195 23139 raft_consensus.cc:2229] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:38.305296 23139 raft_consensus.cc:2258] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:38.306595 23139 ts_tablet_manager.cc:1905] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:38.309054 23139 ts_tablet_manager.cc:1918] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:38.309170 23139 log.cc:1196] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/386c74c3876d42e9b202318b32724209
I20260808 20:56:38.309456 23139 ts_tablet_manager.cc:1939] T 386c74c3876d42e9b202318b32724209 P 117e39e7c2754affaa0402aa336caf90: Deleting consensus metadata
I20260808 20:56:38.309991 22412 catalog_manager.cc:4841] TS 117e39e7c2754affaa0402aa336caf90 (127.15.232.130:46041): tablet 386c74c3876d42e9b202318b32724209 (table test-table [id=efc93bb3c2934ccaa9f60641b23993ba]) successfully deleted
I20260808 20:56:38.310182 23140 ts_tablet_manager.cc:1905] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:38.311882 23140 ts_tablet_manager.cc:1918] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:38.311965 23140 log.cc:1196] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/386c74c3876d42e9b202318b32724209
I20260808 20:56:38.312238 23140 ts_tablet_manager.cc:1939] T 386c74c3876d42e9b202318b32724209 P 19b3152b44c34698bda5d4bff09c1c1c: Deleting consensus metadata
I20260808 20:56:38.312704 22414 catalog_manager.cc:4841] TS 19b3152b44c34698bda5d4bff09c1c1c (127.15.232.131:45061): tablet 386c74c3876d42e9b202318b32724209 (table test-table [id=efc93bb3c2934ccaa9f60641b23993ba]) successfully deleted
W20260808 20:56:38.313686 22414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:35487 (error 108)
W20260808 20:56:38.314316 23097 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:38.314126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35487 (local address 127.0.0.1:47648)
0808 20:56:38.314198 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.314212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.314267 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:35487: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 20:56:38.314460 22414 catalog_manager.cc:4584] TS 71c79bfbc68b419f9c097b6adb8e457b (127.15.232.129:35487): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 386c74c3876d42e9b202318b32724209: Network error: Client connection negotiation failed: client connection to 127.15.232.129:35487: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260808 20:56:38.314769 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 22692
I20260808 20:56:38.326892 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 22891
I20260808 20:56:38.336146 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 22390
2026-08-08T20:56:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2444 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T20:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:38Z Disabled control of system clock
I20260808 20:56:38.372471 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:40699
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:46483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:40699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 20:56:38.458307 23152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:38.458529 23152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:38.458551 23152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:38.459679 23152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:38.459720 23152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:38.459733 23152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:38.459745 23152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:38.459757 23152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:38.461052 23152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:46483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:40699
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:40699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:38.461252 23152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:38.461513 23152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:38.466609 23162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:38.466620 23159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:38.466696 23160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:38.466886 23152 server_base.cc:1034] running on GCE node
I20260808 20:56:38.467209 23152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:38.467576 23152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:38.468786 23152 hybrid_clock.cc:648] HybridClock initialized: now 1786222598468772 us; error 30 us; skew 500 ppm
I20260808 20:56:38.470207 23152 webserver.cc:460] Webserver started at http://127.15.232.190:35067/ using document root <none> and password file <none>
I20260808 20:56:38.470498 23152 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:38.470603 23152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:38.470759 23152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:38.471971 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "3cadd3dbb3084da783a20b8b2db82605"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "60b276322fc066e606915d8ca10860d7"
server_key_iv: "049a0b07339a317cfac023ec5e616435"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.472410 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "3cadd3dbb3084da783a20b8b2db82605"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "60b276322fc066e606915d8ca10860d7"
server_key_iv: "049a0b07339a317cfac023ec5e616435"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.474094 23152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:38.475107 23174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.475414 23152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:38.475633 23152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "3cadd3dbb3084da783a20b8b2db82605"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "60b276322fc066e606915d8ca10860d7"
server_key_iv: "049a0b07339a317cfac023ec5e616435"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.475729 23152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:38.479172 23152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:38.479516 23152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:38.479684 23152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:38.484215 23152 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:40699
I20260808 20:56:38.484246 23243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:40699 every 8 connection(s)
I20260808 20:56:38.484612 23152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:38.485347 23244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:38.487658 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 23152
I20260808 20:56:38.487766 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:38.487984 16290 external_mini_cluster.cc:1428] Setting key 4a985c1805ea4ccc2cbb77a68b224afd
I20260808 20:56:38.488888 23244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605: Bootstrap starting.
I20260808 20:56:38.489568 23244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:38.490299 23244 log.cc:824] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:38.490733 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.488613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.15.232.190:40699)
0808 20:56:38.488802 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.488809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.488834 (+ 25us) server_negotiation.cc:407] Connection header received
0808 20:56:38.489326 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.489333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.489375 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.489497 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.490246 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.490250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.490250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:38.490254 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.490298 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:38.490308 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.490399 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.490458 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.490534 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260808 20:56:38.491101 23244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605: No bootstrap required, opened a new log
I20260808 20:56:38.492615 23244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cadd3dbb3084da783a20b8b2db82605" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40699 } }
I20260808 20:56:38.492756 23244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:38.492779 23244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cadd3dbb3084da783a20b8b2db82605, State: Initialized, Role: FOLLOWER
I20260808 20:56:38.492904 23244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cadd3dbb3084da783a20b8b2db82605" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40699 } }
I20260808 20:56:38.493000 23244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:38.493023 23244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:38.493078 23244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:38.494067 23244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cadd3dbb3084da783a20b8b2db82605" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40699 } }
I20260808 20:56:38.494167 23244 leader_election.cc:302] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cadd3dbb3084da783a20b8b2db82605; no voters:
I20260808 20:56:38.494371 23244 leader_election.cc:288] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:38.494455 23252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:38.494628 23252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [term 1 LEADER]: Becoming Leader. State: Replica: 3cadd3dbb3084da783a20b8b2db82605, State: Running, Role: LEADER
I20260808 20:56:38.494658 23244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:38.494762 23252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cadd3dbb3084da783a20b8b2db82605" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40699 } }
I20260808 20:56:38.495182 23253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cadd3dbb3084da783a20b8b2db82605" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cadd3dbb3084da783a20b8b2db82605" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40699 } } }
I20260808 20:56:38.495177 23255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cadd3dbb3084da783a20b8b2db82605. Latest consensus state: current_term: 1 leader_uuid: "3cadd3dbb3084da783a20b8b2db82605" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cadd3dbb3084da783a20b8b2db82605" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40699 } } }
I20260808 20:56:38.495321 23255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:38.495319 23253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:38.495589 23264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:38.496158 23264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:38.497766 23264 catalog_manager.cc:1347] Generated new cluster ID: 0347bfcdee444742b6f6056392cc08e1
I20260808 20:56:38.497828 23264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:38.508831 23264 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:38.509045 23264 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:38.523504 23264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605: Generated new TSK 0
I20260808 20:56:38.523772 23264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 20:56:38.525385 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40699
--builtin_ntp_servers=127.15.232.148:46483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 20:56:38.611915 23283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:38.612146 23283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:38.612185 23283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:38.613363 23283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:38.613425 23283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:38.613482 23283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:38.614946 23283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:46483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:38.615228 23283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:38.615588 23283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:38.620934 23289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:38.620990 23291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:38.620940 23288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:38.621382 23283 server_base.cc:1034] running on GCE node
I20260808 20:56:38.621596 23283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:38.621863 23283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:38.623044 23283 hybrid_clock.cc:648] HybridClock initialized: now 1786222598623021 us; error 35 us; skew 500 ppm
I20260808 20:56:38.624662 23283 webserver.cc:460] Webserver started at http://127.15.232.129:33939/ using document root <none> and password file <none>
I20260808 20:56:38.624928 23283 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:38.625001 23283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:38.625139 23283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:38.626590 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "69621dc66dc14b1899ed68665ecbcb58"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "5bf2e67da18273447fe71a06e66f6c3e"
server_key_iv: "5260fd146ed3c5bd4e1e4edbac6f363b"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.626998 23283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "69621dc66dc14b1899ed68665ecbcb58"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "5bf2e67da18273447fe71a06e66f6c3e"
server_key_iv: "5260fd146ed3c5bd4e1e4edbac6f363b"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.628638 23283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 20:56:38.629612 23300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.629827 23283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:38.629920 23283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "69621dc66dc14b1899ed68665ecbcb58"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "5bf2e67da18273447fe71a06e66f6c3e"
server_key_iv: "5260fd146ed3c5bd4e1e4edbac6f363b"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.629992 23283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:38.634673 23283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:38.634964 23283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:38.635246 23283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:38.635550 23283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:38.635921 23283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:38.635969 23283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.636000 23283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:38.636026 23283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.642233 23283 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:44551
I20260808 20:56:38.642325 23463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:44551 every 8 connection(s)
I20260808 20:56:38.642666 23283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:38.645987 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.643504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:49899 (local address 127.15.232.190:40699)
0808 20:56:38.643567 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.643569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.643841 (+ 272us) server_negotiation.cc:407] Connection header received
0808 20:56:38.644385 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.644386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.644393 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.644488 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.645717 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.645719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.645719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:38.645724 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.645739 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:38.645743 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.645797 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.645898 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.645913 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 20:56:38.646202 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.643542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:40699 (local address 127.15.232.129:49899)
0808 20:56:38.643704 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.643746 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.644270 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.644540 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.644546 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.644561 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:38.645548 (+ 987us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:56:38.645548 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:38.645578 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:56:38.645607 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:38.645803 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:38.645809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:38.645812 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.645908 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.645979 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260808 20:56:38.646826 23464 heartbeater.cc:347] Connected to a master server at 127.15.232.190:40699
I20260808 20:56:38.646919 23464 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:38.647039 23464 heartbeater.cc:511] Master 127.15.232.190:40699 requested a full tablet report, sending...
I20260808 20:56:38.647617 23195 ts_manager.cc:194] Registered new tserver with Master: 69621dc66dc14b1899ed68665ecbcb58 (127.15.232.129:44551)
I20260808 20:56:38.651552 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 23283
I20260808 20:56:38.651691 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:38.651875 16290 external_mini_cluster.cc:1428] Setting key 71d8cc578ba8596e55cd302ccc454614
I20260808 20:56:38.652788 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40699
--builtin_ntp_servers=127.15.232.148:46483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 20:56:38.738463 23469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:38.738675 23469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:38.738696 23469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:38.739864 23469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:38.739903 23469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:38.739928 23469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:38.741215 23469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:46483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:38.741402 23469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:38.741658 23469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:38.746620 23479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:38.746615 23481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:38.746649 23485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:38.746973 23469 server_base.cc:1034] running on GCE node
I20260808 20:56:38.747242 23469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:38.747555 23469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:38.748713 23469 hybrid_clock.cc:648] HybridClock initialized: now 1786222598748691 us; error 39 us; skew 500 ppm
I20260808 20:56:38.750020 23469 webserver.cc:460] Webserver started at http://127.15.232.130:46773/ using document root <none> and password file <none>
I20260808 20:56:38.750298 23469 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:38.750368 23469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:38.750512 23469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:38.751703 23469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "aeb20900ca3e4f039a5a4d3410bf3874"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "4d97baf7ab3165ef7b88a9002f6b876e"
server_key_iv: "264e81ba35132ca36378498c7a872abc"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.752092 23469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "aeb20900ca3e4f039a5a4d3410bf3874"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "4d97baf7ab3165ef7b88a9002f6b876e"
server_key_iv: "264e81ba35132ca36378498c7a872abc"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.753662 23469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:38.754455 23492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.754649 23469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:38.754745 23469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "aeb20900ca3e4f039a5a4d3410bf3874"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "4d97baf7ab3165ef7b88a9002f6b876e"
server_key_iv: "264e81ba35132ca36378498c7a872abc"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.754832 23469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:38.758410 23469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:38.758693 23469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:38.758836 23469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:38.759057 23469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:38.759377 23469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:38.759431 23469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.759478 23469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:38.759506 23469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.765764 23469 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:45021
I20260808 20:56:38.765806 23650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:45021 every 8 connection(s)
I20260808 20:56:38.766172 23469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:38.768601 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 23469
I20260808 20:56:38.768734 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:38.768944 16290 external_mini_cluster.cc:1428] Setting key 67bd90dd811b4fc551a2832a0541ad44
I20260808 20:56:38.769630 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.766966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:40383 (local address 127.15.232.190:40699)
0808 20:56:38.767042 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.767044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.767306 (+ 262us) server_negotiation.cc:407] Connection header received
0808 20:56:38.767875 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.767876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.767882 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.767952 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.769368 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.769370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.769371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:38.769376 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.769391 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:38.769395 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.769464 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.769535 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.769548 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 20:56:38.769842 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.766944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:40699 (local address 127.15.232.130:40383)
0808 20:56:38.767166 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.767209 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.767770 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.768025 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.768029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.768038 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:38.769175 (+ 1137us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:56:38.769176 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:38.769205 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:56:38.769255 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:38.769464 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:38.769467 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:38.769470 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.769526 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.769578 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260808 20:56:38.770284 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40699
--builtin_ntp_servers=127.15.232.148:46483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 20:56:38.770326 23651 heartbeater.cc:347] Connected to a master server at 127.15.232.190:40699
I20260808 20:56:38.770419 23651 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:38.770571 23651 heartbeater.cc:511] Master 127.15.232.190:40699 requested a full tablet report, sending...
I20260808 20:56:38.770969 23195 ts_manager.cc:194] Registered new tserver with Master: aeb20900ca3e4f039a5a4d3410bf3874 (127.15.232.130:45021)
W20260808 20:56:38.853853 23655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:38.854136 23655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:38.854177 23655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:38.855394 23655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:38.855486 23655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:38.855546 23655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:38.856935 23655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:46483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:38.857216 23655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:38.857594 23655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:38.863047 23664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:38.863111 23669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:38.863049 23662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:38.863263 23655 server_base.cc:1034] running on GCE node
I20260808 20:56:38.863636 23655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:38.863902 23655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:38.865072 23655 hybrid_clock.cc:648] HybridClock initialized: now 1786222598865048 us; error 43 us; skew 500 ppm
I20260808 20:56:38.866439 23655 webserver.cc:460] Webserver started at http://127.15.232.131:40731/ using document root <none> and password file <none>
I20260808 20:56:38.866714 23655 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:38.866787 23655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:38.866961 23655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:38.868248 23655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "df97dade12be44439cf5a8e4cf95f29f"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "250f71df20b87acb14415eacf62bea10"
server_key_iv: "9595e0067a8c73487c4f781ef66a3b7b"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.868637 23655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "df97dade12be44439cf5a8e4cf95f29f"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "250f71df20b87acb14415eacf62bea10"
server_key_iv: "9595e0067a8c73487c4f781ef66a3b7b"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.870292 23655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 20:56:38.871170 23677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.871372 23655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:38.871502 23655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "df97dade12be44439cf5a8e4cf95f29f"
format_stamp: "Formatted at 2026-08-08 20:56:38 on dist-test-slave-c12x"
server_key: "250f71df20b87acb14415eacf62bea10"
server_key_iv: "9595e0067a8c73487c4f781ef66a3b7b"
server_key_version: "encryptionkey@0"
I20260808 20:56:38.871600 23655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:38.875321 23655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:38.875649 23655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:38.875801 23655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:38.876052 23655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:38.876406 23655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:38.876462 23655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.876498 23655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:38.876524 23655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:38.882784 23655 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:33363
I20260808 20:56:38.882828 23855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:33363 every 8 connection(s)
I20260808 20:56:38.883198 23655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:38.886348 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 23655
I20260808 20:56:38.886463 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:38.886531 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.883971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:40375 (local address 127.15.232.190:40699)
0808 20:56:38.884062 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.884064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.884266 (+ 202us) server_negotiation.cc:407] Connection header received
0808 20:56:38.884800 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.884801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.884807 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.884870 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.886138 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.886140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.886140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:38.886145 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.886161 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:38.886165 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.886216 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.886366 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.886382 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 20:56:38.886675 16290 external_mini_cluster.cc:1428] Setting key 0f255bf50a9250e13e6b7486dc01c03a
I20260808 20:56:38.886735 23860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.883922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:40699 (local address 127.15.232.131:40375)
0808 20:56:38.884135 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.884170 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.884636 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.884993 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.884996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.885005 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:38.885957 (+ 952us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:56:38.885958 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:38.885986 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:56:38.886016 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:38.886238 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:38.886244 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:38.886247 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.886342 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.886414 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260808 20:56:38.887203 23856 heartbeater.cc:347] Connected to a master server at 127.15.232.190:40699
I20260808 20:56:38.887310 23856 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:38.887513 23856 heartbeater.cc:511] Master 127.15.232.190:40699 requested a full tablet report, sending...
I20260808 20:56:38.887903 23191 ts_manager.cc:194] Registered new tserver with Master: df97dade12be44439cf5a8e4cf95f29f (127.15.232.131:33363)
I20260808 20:56:38.898886 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:56:38.901922 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.900458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.15.232.190:40699)
0808 20:56:38.900534 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.900536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.900663 (+ 127us) server_negotiation.cc:407] Connection header received
0808 20:56:38.900876 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.900878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.900885 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.900944 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.901621 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.901622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.901623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:38.901626 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.901640 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:38.901644 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.901705 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.901783 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.901796 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 20:56:38.903879 23191 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:38.905526 23191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:38.909937 23873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.908360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.15.232.131:33363)
0808 20:56:38.908705 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.908710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.908723 (+ 13us) server_negotiation.cc:407] Connection header received
0808 20:56:38.908754 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.908757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.908769 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.909135 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.909601 (+ 466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.909610 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.909610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:38.909614 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.909648 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:38.909670 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.909761 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.909839 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.909855 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":91,"threads_started":1}
I20260808 20:56:38.909931 23870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.908165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:33363 (local address 127.0.0.1:44976)
0808 20:56:38.908350 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.908389 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.908447 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.908873 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.908880 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.908887 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:38.909454 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:56:38.909455 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:38.909472 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:56:38.909510 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:38.909758 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:38.909766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:38.909769 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.909833 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.909848 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
I20260808 20:56:38.910084 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.908166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45021 (local address 127.0.0.1:37620)
0808 20:56:38.908350 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.908388 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.908447 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.909094 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.909094 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.909108 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:38.909604 (+ 496us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:56:38.909605 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:38.909614 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:56:38.909627 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:38.909873 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:38.909874 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:38.909875 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.909909 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.909924 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":4,"spinlock_wait_cycles":384,"thread_start_us":97,"threads_started":1}
I20260808 20:56:38.910152 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.908555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.15.232.130:45021)
0808 20:56:38.908746 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.908750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.908760 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:38.908791 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.908792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.908818 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.908933 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.909707 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.909714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.909715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:38.909719 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.909760 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:38.909783 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.909999 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.910024 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.910040 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
I20260808 20:56:38.910907 23564 tablet_service.cc:1467] Processing CreateTablet for tablet 466a3b5af03c46939dd24b27fb23bfe8 (DEFAULT_TABLE table=test-table [id=f50e67db97b54397a64de782d3feb594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:38.910908 23754 tablet_service.cc:1467] Processing CreateTablet for tablet 466a3b5af03c46939dd24b27fb23bfe8 (DEFAULT_TABLE table=test-table [id=f50e67db97b54397a64de782d3feb594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:38.911325 23754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 466a3b5af03c46939dd24b27fb23bfe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:38.911365 23564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 466a3b5af03c46939dd24b27fb23bfe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:38.911804 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.908482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49370 (local address 127.15.232.129:44551)
0808 20:56:38.908691 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:38.908696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:38.908950 (+ 254us) server_negotiation.cc:407] Connection header received
0808 20:56:38.908981 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:38.908984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:38.908993 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:38.909709 (+ 716us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:38.911159 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:38.911168 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:38.911168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:38.911174 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:38.911209 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:38.911239 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:38.911330 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:38.911676 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 20:56:38.911694 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260808 20:56:38.911975 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:38.908173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:44551 (local address 127.0.0.1:49370)
0808 20:56:38.908741 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:38.908751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:38.908815 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:38.910524 (+ 1709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:38.910526 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:38.910529 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:38.910984 (+ 455us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:56:38.910985 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:38.910992 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:56:38.911012 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:38.911549 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:38.911550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:38.911551 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 20:56:38.911602 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 20:56:38.911910 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":129,"threads_started":1}
I20260808 20:56:38.912726 23377 tablet_service.cc:1467] Processing CreateTablet for tablet 466a3b5af03c46939dd24b27fb23bfe8 (DEFAULT_TABLE table=test-table [id=f50e67db97b54397a64de782d3feb594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:38.913103 23377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 466a3b5af03c46939dd24b27fb23bfe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:38.914363 23878 tablet_bootstrap.cc:492] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: Bootstrap starting.
I20260808 20:56:38.914765 23879 tablet_bootstrap.cc:492] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: Bootstrap starting.
I20260808 20:56:38.915242 23878 tablet_bootstrap.cc:654] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:38.915616 23879 tablet_bootstrap.cc:654] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:38.915656 23878 log.cc:824] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:38.915938 23879 log.cc:824] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:38.916453 23883 tablet_bootstrap.cc:492] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58: Bootstrap starting.
I20260808 20:56:38.916551 23878 tablet_bootstrap.cc:492] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: No bootstrap required, opened a new log
I20260808 20:56:38.916620 23878 ts_tablet_manager.cc:1397] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 20:56:38.917178 23883 tablet_bootstrap.cc:654] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:38.917483 23883 log.cc:824] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:38.918196 23883 tablet_bootstrap.cc:492] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58: No bootstrap required, opened a new log
I20260808 20:56:38.918280 23883 ts_tablet_manager.cc:1397] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:38.918742 23878 raft_consensus.cc:348] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } }
I20260808 20:56:38.918877 23878 raft_consensus.cc:374] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:38.918898 23878 raft_consensus.cc:729] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeb20900ca3e4f039a5a4d3410bf3874, State: Initialized, Role: FOLLOWER
I20260808 20:56:38.919035 23878 consensus_queue.cc:260] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } }
I20260808 20:56:38.919307 23878 ts_tablet_manager.cc:1428] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 20:56:38.919368 23651 heartbeater.cc:503] Master 127.15.232.190:40699 was elected leader, sending a full tablet report...
I20260808 20:56:38.920085 23879 tablet_bootstrap.cc:492] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: No bootstrap required, opened a new log
I20260808 20:56:38.920223 23879 ts_tablet_manager.cc:1397] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260808 20:56:38.921423 23883 raft_consensus.cc:348] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } }
I20260808 20:56:38.921620 23883 raft_consensus.cc:374] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:38.921661 23883 raft_consensus.cc:729] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69621dc66dc14b1899ed68665ecbcb58, State: Initialized, Role: FOLLOWER
I20260808 20:56:38.921801 23883 consensus_queue.cc:260] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } }
I20260808 20:56:38.922093 23883 ts_tablet_manager.cc:1428] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 20:56:38.922147 23464 heartbeater.cc:503] Master 127.15.232.190:40699 was elected leader, sending a full tablet report...
I20260808 20:56:38.923040 23879 raft_consensus.cc:348] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } }
I20260808 20:56:38.923202 23879 raft_consensus.cc:374] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:38.923251 23879 raft_consensus.cc:729] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df97dade12be44439cf5a8e4cf95f29f, State: Initialized, Role: FOLLOWER
I20260808 20:56:38.923368 23879 consensus_queue.cc:260] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } }
I20260808 20:56:38.923661 23879 ts_tablet_manager.cc:1428] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260808 20:56:38.923731 23856 heartbeater.cc:503] Master 127.15.232.190:40699 was elected leader, sending a full tablet report...
I20260808 20:56:39.007673 23886 raft_consensus.cc:482] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:39.007819 23886 raft_consensus.cc:504] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } }
I20260808 20:56:39.008291 23886 leader_election.cc:288] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df97dade12be44439cf5a8e4cf95f29f (127.15.232.131:33363), 69621dc66dc14b1899ed68665ecbcb58 (127.15.232.129:44551)
I20260808 20:56:39.009855 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.008530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:44551 (local address 127.15.232.130:40191)
0808 20:56:39.008595 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:39.008609 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:39.008651 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:39.008838 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:39.008839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:39.008841 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:39.009381 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:56:39.009382 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:39.009391 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:56:39.009406 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:39.009696 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:39.009697 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:39.009698 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 20:56:39.009754 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 20:56:39.009770 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 20:56:39.009909 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.008635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:40191 (local address 127.15.232.129:44551)
0808 20:56:39.008686 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.008689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.008701 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:56:39.008734 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.008735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.008742 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.008819 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:39.009521 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:39.009523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:39.009524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:39.009529 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:39.009581 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:39.009587 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:39.009660 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:39.009769 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 20:56:39.009804 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:56:39.009972 23873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.008627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:59835 (local address 127.15.232.131:33363)
0808 20:56:39.008714 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.008716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.008817 (+ 101us) server_negotiation.cc:407] Connection header received
0808 20:56:39.008864 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.008865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.008872 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.008942 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:39.009731 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:39.009732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:39.009733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:39.009738 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:39.009754 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:39.009760 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:39.009817 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:39.009879 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 20:56:39.009892 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 20:56:39.010165 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.008531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:33363 (local address 127.15.232.130:59835)
0808 20:56:39.008729 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:39.008740 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:39.008805 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:39.008954 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:39.008954 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:39.008956 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:39.009616 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:56:39.009617 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:39.009625 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:56:39.009638 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:39.009825 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:39.009826 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:39.009826 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 20:56:39.009881 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 20:56:39.009894 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":16,"thread_start_us":105,"threads_started":1}
I20260808 20:56:39.010281 23404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "466a3b5af03c46939dd24b27fb23bfe8" candidate_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69621dc66dc14b1899ed68665ecbcb58" is_pre_election: true
I20260808 20:56:39.010505 23404 raft_consensus.cc:2454] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeb20900ca3e4f039a5a4d3410bf3874 in term 0.
I20260808 20:56:39.010638 23784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "466a3b5af03c46939dd24b27fb23bfe8" candidate_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df97dade12be44439cf5a8e4cf95f29f" is_pre_election: true
I20260808 20:56:39.010790 23784 raft_consensus.cc:2454] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aeb20900ca3e4f039a5a4d3410bf3874 in term 0.
I20260808 20:56:39.010865 23498 leader_election.cc:302] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69621dc66dc14b1899ed68665ecbcb58, aeb20900ca3e4f039a5a4d3410bf3874; no voters:
I20260808 20:56:39.011097 23886 raft_consensus.cc:2781] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:39.011148 23886 raft_consensus.cc:482] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:39.011165 23886 raft_consensus.cc:3037] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:39.011962 23886 raft_consensus.cc:504] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } }
I20260808 20:56:39.012115 23886 leader_election.cc:288] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [CANDIDATE]: Term 1 election: Requested vote from peers df97dade12be44439cf5a8e4cf95f29f (127.15.232.131:33363), 69621dc66dc14b1899ed68665ecbcb58 (127.15.232.129:44551)
I20260808 20:56:39.012277 23784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "466a3b5af03c46939dd24b27fb23bfe8" candidate_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df97dade12be44439cf5a8e4cf95f29f"
I20260808 20:56:39.012293 23404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "466a3b5af03c46939dd24b27fb23bfe8" candidate_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69621dc66dc14b1899ed68665ecbcb58"
I20260808 20:56:39.012384 23784 raft_consensus.cc:3037] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:39.012379 23404 raft_consensus.cc:3037] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:39.013113 23784 raft_consensus.cc:2454] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeb20900ca3e4f039a5a4d3410bf3874 in term 1.
I20260808 20:56:39.013156 23404 raft_consensus.cc:2454] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aeb20900ca3e4f039a5a4d3410bf3874 in term 1.
I20260808 20:56:39.013322 23498 leader_election.cc:302] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69621dc66dc14b1899ed68665ecbcb58, aeb20900ca3e4f039a5a4d3410bf3874; no voters:
I20260808 20:56:39.013417 23886 raft_consensus.cc:2781] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:39.013621 23886 raft_consensus.cc:686] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 1 LEADER]: Becoming Leader. State: Replica: aeb20900ca3e4f039a5a4d3410bf3874, State: Running, Role: LEADER
I20260808 20:56:39.013734 23886 consensus_queue.cc:237] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } }
I20260808 20:56:39.014684 23191 catalog_manager.cc:5495] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 reported cstate change: term changed from 0 to 1, leader changed from <none> to aeb20900ca3e4f039a5a4d3410bf3874 (127.15.232.130). New cstate: current_term: 1 leader_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeb20900ca3e4f039a5a4d3410bf3874" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 } health_report { overall_health: UNKNOWN } } }
W20260808 20:56:39.016691 23653 tablet.cc:2369] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:39.029166 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.027607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.15.232.130:45021)
0808 20:56:39.027661 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.027663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.027674 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:39.027738 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.027739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.027752 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.027815 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:39.028655 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:39.028657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:39.028657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:39.028661 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:39.028678 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:39.028682 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:39.028744 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:39.029015 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 20:56:39.029036 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:56:39.031603 23404 raft_consensus.cc:1261] T 466a3b5af03c46939dd24b27fb23bfe8 P 69621dc66dc14b1899ed68665ecbcb58 [term 1 FOLLOWER]: Refusing update from remote peer aeb20900ca3e4f039a5a4d3410bf3874: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:39.031613 23784 raft_consensus.cc:1261] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [term 1 FOLLOWER]: Refusing update from remote peer aeb20900ca3e4f039a5a4d3410bf3874: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:39.032048 23886 consensus_queue.cc:1035] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69621dc66dc14b1899ed68665ecbcb58" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:39.032240 23893 consensus_queue.cc:1035] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df97dade12be44439cf5a8e4cf95f29f" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 33363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:39.035238 23900 mvcc.cc:205] Tried to move back new op lower bound from 7316367765629460480 to 7316367765560877056. Current Snapshot: MvccSnapshot[applied={T|T < 7316367765629460480}]
I20260808 20:56:39.035528 23901 mvcc.cc:205] Tried to move back new op lower bound from 7316367765629460480 to 7316367765560877056. Current Snapshot: MvccSnapshot[applied={T|T < 7316367765629460480}]
I20260808 20:56:39.035796 23902 mvcc.cc:205] Tried to move back new op lower bound from 7316367765629460480 to 7316367765560877056. Current Snapshot: MvccSnapshot[applied={T|T < 7316367765629460480}]
I20260808 20:56:39.038180 23191 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:39.038277 23191 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:39.039177 23191 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cadd3dbb3084da783a20b8b2db82605: Sending DeleteTablet for 3 replicas of tablet 466a3b5af03c46939dd24b27fb23bfe8
I20260808 20:56:39.039599 23564 tablet_service.cc:1514] Processing DeleteTablet for tablet 466a3b5af03c46939dd24b27fb23bfe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:39 UTC) from {username='slave'} at 127.0.0.1:37620
I20260808 20:56:39.039690 23754 tablet_service.cc:1514] Processing DeleteTablet for tablet 466a3b5af03c46939dd24b27fb23bfe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:39 UTC) from {username='slave'} at 127.0.0.1:44976
I20260808 20:56:39.039762 23377 tablet_service.cc:1514] Processing DeleteTablet for tablet 466a3b5af03c46939dd24b27fb23bfe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:39 UTC) from {username='slave'} at 127.0.0.1:49370
I20260808 20:56:39.039971 23913 tablet_replica.cc:330] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: stopping tablet replica
I20260808 20:56:39.040010 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 23283
I20260808 20:56:39.040081 23913 raft_consensus.cc:2229] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:39.040239 23913 raft_consensus.cc:2258] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:39.040776 23913 ts_tablet_manager.cc:1905] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:39.042348 23914 tablet_replica.cc:330] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: stopping tablet replica
I20260808 20:56:39.042471 23914 raft_consensus.cc:2229] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:39.042587 23914 raft_consensus.cc:2258] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:39.044565 23914 ts_tablet_manager.cc:1905] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:39.046587 23914 ts_tablet_manager.cc:1918] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:39.046670 23914 log.cc:1196] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/466a3b5af03c46939dd24b27fb23bfe8
I20260808 20:56:39.046903 23914 ts_tablet_manager.cc:1939] T 466a3b5af03c46939dd24b27fb23bfe8 P df97dade12be44439cf5a8e4cf95f29f: Deleting consensus metadata
I20260808 20:56:39.047410 23175 catalog_manager.cc:4841] TS df97dade12be44439cf5a8e4cf95f29f (127.15.232.131:33363): tablet 466a3b5af03c46939dd24b27fb23bfe8 (table test-table [id=f50e67db97b54397a64de782d3feb594]) successfully deleted
I20260808 20:56:39.055701 23913 ts_tablet_manager.cc:1918] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:39.055826 23913 log.cc:1196] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/466a3b5af03c46939dd24b27fb23bfe8
I20260808 20:56:39.056173 23913 ts_tablet_manager.cc:1939] T 466a3b5af03c46939dd24b27fb23bfe8 P aeb20900ca3e4f039a5a4d3410bf3874: Deleting consensus metadata
I20260808 20:56:39.058262 23176 catalog_manager.cc:4841] TS aeb20900ca3e4f039a5a4d3410bf3874 (127.15.232.130:45021): tablet 466a3b5af03c46939dd24b27fb23bfe8 (table test-table [id=f50e67db97b54397a64de782d3feb594]) successfully deleted
W20260808 20:56:39.062937 23179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:44551 (error 108)
I20260808 20:56:39.063153 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 23469
I20260808 20:56:39.066201 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.066040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:44551 (local address 127.0.0.1:49384)
0808 20:56:39.066102 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:39.066151 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:44551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 20:56:39.066365 23179 catalog_manager.cc:4584] TS 69621dc66dc14b1899ed68665ecbcb58 (127.15.232.129:44551): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 466a3b5af03c46939dd24b27fb23bfe8: Network error: Client connection negotiation failed: client connection to 127.15.232.129:44551: connect: Connection refused (error 111)
I20260808 20:56:39.070533 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 23655
I20260808 20:56:39.079627 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 23152
2026-08-08T20:56:39Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (744 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 20:56:39.108364 16290 mini_oidc.cc:152] Starting JWKS server
I20260808 20:56:39.109652 16290 webserver.cc:460] Webserver started at https://127.0.0.1:40803/ using document root <none> and password file <none>
I20260808 20:56:39.109784 16290 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 20:56:39.109984 16290 webserver.cc:460] Webserver started at http://127.0.0.1:40617/ using document root <none> and password file <none>
2026-08-08T20:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:39Z Disabled control of system clock
I20260808 20:56:39.119783 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33939
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:33939
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40803/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/testchainca.cert with env {}
W20260808 20:56:39.203549 23927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:39.203780 23927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:39.203804 23927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:39.204963 23927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:39.205008 23927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:39.205021 23927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:39.205034 23927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:39.205045 23927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 20:56:39.205057 23927 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 20:56:39.205068 23927 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40803/jwks.json
I20260808 20:56:39.206581 23927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:33939
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40803/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:39.206792 23927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:39.206954 23927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:39.212059 23933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:39.212087 23937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:39.212064 23935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:39.212373 23927 server_base.cc:1034] running on GCE node
I20260808 20:56:39.212621 23927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:39.212889 23927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:39.214064 23927 hybrid_clock.cc:648] HybridClock initialized: now 1786222599214048 us; error 29 us; skew 500 ppm
I20260808 20:56:39.215503 23927 webserver.cc:460] Webserver started at http://127.15.232.190:39615/ using document root <none> and password file <none>
I20260808 20:56:39.215757 23927 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:39.215807 23927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:39.215969 23927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:39.217176 23927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "172370a34c134b65bb2c601d44cd5505"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "d5a69224c08759fbc9f05d2cd432b516"
server_key_iv: "1f1831bfccf0a671fdaecacf6353fc2f"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.217633 23927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "172370a34c134b65bb2c601d44cd5505"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "d5a69224c08759fbc9f05d2cd432b516"
server_key_iv: "1f1831bfccf0a671fdaecacf6353fc2f"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.219226 23927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:39.220130 23943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:39.220311 23927 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:39.220409 23927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "172370a34c134b65bb2c601d44cd5505"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "d5a69224c08759fbc9f05d2cd432b516"
server_key_iv: "1f1831bfccf0a671fdaecacf6353fc2f"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.220510 23927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:39.242239 23927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:39.242636 23927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:39.242794 23927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:39.247390 23927 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:33939
I20260808 20:56:39.247540 24015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:33939 every 8 connection(s)
I20260808 20:56:39.247931 23927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:39.249054 24017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:39.251628 24017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505: Bootstrap starting.
I20260808 20:56:39.252305 24017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:39.252626 24017 log.cc:824] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:39.253365 24017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505: No bootstrap required, opened a new log
I20260808 20:56:39.255067 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 23927
I20260808 20:56:39.255182 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:39.255188 24017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172370a34c134b65bb2c601d44cd5505" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33939 } }
I20260808 20:56:39.255383 16290 external_mini_cluster.cc:1428] Setting key ff8cb80eeaad73d1e3da7706fe189f3c
I20260808 20:56:39.255380 24017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:39.255422 24017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 172370a34c134b65bb2c601d44cd5505, State: Initialized, Role: FOLLOWER
I20260808 20:56:39.255609 24017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172370a34c134b65bb2c601d44cd5505" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33939 } }
I20260808 20:56:39.255720 24017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:39.255765 24017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:39.255825 24017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:39.256510 24017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172370a34c134b65bb2c601d44cd5505" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33939 } }
I20260808 20:56:39.256644 24017 leader_election.cc:302] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 172370a34c134b65bb2c601d44cd5505; no voters:
I20260808 20:56:39.256819 24017 leader_election.cc:288] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:39.257073 24028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:39.257097 24017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:39.257385 24028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [term 1 LEADER]: Becoming Leader. State: Replica: 172370a34c134b65bb2c601d44cd5505, State: Running, Role: LEADER
I20260808 20:56:39.257530 24028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172370a34c134b65bb2c601d44cd5505" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33939 } }
I20260808 20:56:39.257942 24026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "172370a34c134b65bb2c601d44cd5505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172370a34c134b65bb2c601d44cd5505" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33939 } } }
I20260808 20:56:39.257948 24029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 172370a34c134b65bb2c601d44cd5505. Latest consensus state: current_term: 1 leader_uuid: "172370a34c134b65bb2c601d44cd5505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "172370a34c134b65bb2c601d44cd5505" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33939 } } }
I20260808 20:56:39.258035 24026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:39.258056 24029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:39.258230 24041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:39.258720 24041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:39.260607 24041 catalog_manager.cc:1347] Generated new cluster ID: f4d7e60cbbbf417c8ec0e170a4194c19
I20260808 20:56:39.260659 24041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:39.262178 24025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.255985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.15.232.190:33939)
0808 20:56:39.256244 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.256253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.256287 (+ 34us) server_negotiation.cc:407] Connection header received
0808 20:56:39.256387 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.256393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.256471 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.256582 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:39.256852 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.258733 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.259870 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.260542 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.260664 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.261447 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:39.261451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:39.261451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:39.261653 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:39.261691 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:39.261705 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:39.261772 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:39.261872 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 20:56:39.261953 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":127,"threads_started":1}
I20260808 20:56:39.263109 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 20:56:39.280509 24041 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:39.281273 24041 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:39.291121 24041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 172370a34c134b65bb2c601d44cd5505: Generated new TSK 0
I20260808 20:56:39.291370 24041 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 20:56:39.323954 24068 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 20:56:39.324121 24068 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260808 20:56:39.324717 24067 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:39.299723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33939 (local address 127.0.0.1:52390)
0808 20:56:39.299935 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:39.299948 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:39.300028 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:39.300245 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:39.300246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:39.300248 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 20:56:39.300320 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:39.300322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.301086 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.301087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:39.302053 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:39.302057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.302471 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.302472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:39.302541 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.302565 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 20:56:39.324201 (+ 21636us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40803/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 20:56:39.324341 (+ 140us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.232.190:33939: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40803/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":118,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.232.190:33939: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40803/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 20:56:39.325228 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 23927
I20260808 20:56:39.325238 24025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.299792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52390 (local address 127.15.232.190:33939)
0808 20:56:39.299866 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.299868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.300023 (+ 155us) server_negotiation.cc:407] Connection header received
0808 20:56:39.300131 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.300132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.300154 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.300215 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0808 20:56:39.300432 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.301000 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.302166 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.302369 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.302444 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.302683 (+ 239us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 20:56:39.323952 (+ 21269us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40803/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 20:56:39.325137 (+ 1185us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52390: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40803/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":45}
2026-08-08T20:56:39Z chronyd exiting
I20260808 20:56:39.367344 16290 test_util.cc:175] -----------------------------------------------
I20260808 20:56:39.367426 16290 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786222573557010-16290-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (260 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 20:56:39.368405 16290 mini_oidc.cc:152] Starting JWKS server
I20260808 20:56:39.369153 16290 webserver.cc:460] Webserver started at https://127.0.0.1:43463/ using document root <none> and password file <none>
I20260808 20:56:39.369226 16290 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 20:56:39.369408 16290 webserver.cc:460] Webserver started at http://127.0.0.1:41283/ using document root <none> and password file <none>
2026-08-08T20:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:39Z Disabled control of system clock
I20260808 20:56:39.378463 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35947
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:35947
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43463/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/testchainca.cert with env {}
W20260808 20:56:39.464092 24077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:39.464361 24077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:39.464398 24077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:39.465636 24077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:39.465708 24077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:39.465730 24077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:39.465754 24077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:39.465782 24077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 20:56:39.465812 24077 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 20:56:39.465833 24077 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43463/jwks.json
I20260808 20:56:39.467413 24077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:35947
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43463/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:39.467757 24077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:39.467990 24077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:39.473335 24086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:39.473335 24091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:39.473372 24084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:39.473637 24077 server_base.cc:1034] running on GCE node
I20260808 20:56:39.473897 24077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:39.474201 24077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:39.475370 24077 hybrid_clock.cc:648] HybridClock initialized: now 1786222599475346 us; error 43 us; skew 500 ppm
I20260808 20:56:39.476769 24077 webserver.cc:460] Webserver started at http://127.15.232.190:33025/ using document root <none> and password file <none>
I20260808 20:56:39.477015 24077 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:39.477087 24077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:39.477306 24077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:39.478610 24077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "9ed53053de044cd384d51bb8b647957f"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "b0d63c92bafa72e3199355f41c7e24da"
server_key_iv: "763e3cc992bb10883af4137e86cfdc0f"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.479067 24077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "9ed53053de044cd384d51bb8b647957f"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "b0d63c92bafa72e3199355f41c7e24da"
server_key_iv: "763e3cc992bb10883af4137e86cfdc0f"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.480686 24077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260808 20:56:39.481477 24098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:39.481664 24077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:39.481776 24077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "9ed53053de044cd384d51bb8b647957f"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "b0d63c92bafa72e3199355f41c7e24da"
server_key_iv: "763e3cc992bb10883af4137e86cfdc0f"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.481854 24077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:39.502564 24077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:39.502943 24077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:39.503110 24077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:39.507412 24077 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:35947
I20260808 20:56:39.507527 24167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:35947 every 8 connection(s)
I20260808 20:56:39.507905 24077 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:39.508656 24168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:39.511682 24168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f: Bootstrap starting.
I20260808 20:56:39.512398 24168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:39.512704 24168 log.cc:824] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:39.513566 24168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f: No bootstrap required, opened a new log
I20260808 20:56:39.513700 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 24077
I20260808 20:56:39.513820 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:39.514019 16290 external_mini_cluster.cc:1428] Setting key 9afc16b890d058c933b97fde36540ef0
I20260808 20:56:39.515983 24168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed53053de044cd384d51bb8b647957f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35947 } }
I20260808 20:56:39.516171 24168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:39.516212 24168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ed53053de044cd384d51bb8b647957f, State: Initialized, Role: FOLLOWER
I20260808 20:56:39.516348 24168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed53053de044cd384d51bb8b647957f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35947 } }
I20260808 20:56:39.516474 24168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:39.516506 24168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:39.516554 24168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:39.517359 24168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed53053de044cd384d51bb8b647957f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35947 } }
I20260808 20:56:39.517472 24168 leader_election.cc:302] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ed53053de044cd384d51bb8b647957f; no voters:
I20260808 20:56:39.517642 24168 leader_election.cc:288] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:39.517704 24174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:39.517891 24174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [term 1 LEADER]: Becoming Leader. State: Replica: 9ed53053de044cd384d51bb8b647957f, State: Running, Role: LEADER
I20260808 20:56:39.517894 24168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:39.518021 24174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed53053de044cd384d51bb8b647957f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35947 } }
I20260808 20:56:39.518369 24176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ed53053de044cd384d51bb8b647957f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed53053de044cd384d51bb8b647957f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35947 } } }
I20260808 20:56:39.518356 24174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ed53053de044cd384d51bb8b647957f. Latest consensus state: current_term: 1 leader_uuid: "9ed53053de044cd384d51bb8b647957f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed53053de044cd384d51bb8b647957f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35947 } } }
I20260808 20:56:39.518484 24176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:39.518505 24174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:39.519603 24184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:39.520363 24184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:39.521823 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.514606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.15.232.190:35947)
0808 20:56:39.514852 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.514862 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.514897 (+ 35us) server_negotiation.cc:407] Connection header received
0808 20:56:39.515933 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.515943 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.516038 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.516190 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:39.516458 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.518309 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.519537 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.520178 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.520307 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.521109 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:39.521113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:39.521114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:39.521299 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:39.521341 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:39.521352 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:39.521437 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:39.521517 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 20:56:39.521601 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260808 20:56:39.522244 24184 catalog_manager.cc:1347] Generated new cluster ID: 6dd05ec81ffe4ed880f36d100576e3f6
I20260808 20:56:39.522302 24184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:39.522706 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 20:56:39.540939 24184 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:39.541710 24184 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:39.545529 24184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ed53053de044cd384d51bb8b647957f: Generated new TSK 0
I20260808 20:56:39.545727 24184 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 20:56:39.567466 24211 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 20:56:39.567584 24211 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 20:56:39.567958 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.553010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.15.232.190:35947)
0808 20:56:39.553080 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.553082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.553130 (+ 48us) server_negotiation.cc:407] Connection header received
0808 20:56:39.553207 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.553208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.553219 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.553278 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0808 20:56:39.553432 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.553938 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.555160 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.555356 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.555460 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.555664 (+ 204us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 20:56:39.567733 (+ 12069us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43463/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 20:56:39.567901 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34856: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43463/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 20:56:39.568044 24173 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34856: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43463/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 20:56:39.568158 24210 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:39.552863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35947 (local address 127.0.0.1:34856)
0808 20:56:39.553056 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:39.553068 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:39.553138 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:39.553296 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:39.553297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:39.553299 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 20:56:39.553360 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:39.553363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.554026 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.554027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:39.555048 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:39.555051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.555475 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.555476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:39.555541 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.555548 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 20:56:39.567868 (+ 12320us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43463/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 20:56:39.568052 (+ 184us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.232.190:35947: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43463/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":119,"threads_started":1}
I20260808 20:56:39.568370 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 24077
2026-08-08T20:56:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (246 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 20:56:39.615833 16290 mini_oidc.cc:152] Starting JWKS server
I20260808 20:56:39.616978 16290 webserver.cc:460] Webserver started at https://127.0.0.1:37253/ using document root <none> and password file <none>
I20260808 20:56:39.617051 16290 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 20:56:39.617233 16290 webserver.cc:460] Webserver started at http://127.0.0.1:46553/ using document root <none> and password file <none>
2026-08-08T20:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:39Z Disabled control of system clock
I20260808 20:56:39.628499 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:34745
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:34745
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37253/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 20:56:39.713485 24217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:39.713712 24217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:39.713737 24217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:39.714854 24217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:39.714890 24217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:39.714903 24217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:39.714915 24217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:39.714927 24217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 20:56:39.714939 24217 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 20:56:39.714951 24217 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37253/jwks.json
I20260808 20:56:39.716387 24217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:34745
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:34745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37253/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:39.716619 24217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:39.716866 24217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 20:56:39.722286 24217 server_base.cc:1034] running on GCE node
W20260808 20:56:39.722153 24229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:39.722159 24224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:39.722208 24225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:39.722640 24217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:39.723009 24217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:39.724162 24217 hybrid_clock.cc:648] HybridClock initialized: now 1786222599724147 us; error 31 us; skew 500 ppm
I20260808 20:56:39.725571 24217 webserver.cc:460] Webserver started at http://127.15.232.190:42857/ using document root <none> and password file <none>
I20260808 20:56:39.725847 24217 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:39.725898 24217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:39.726033 24217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:39.727262 24217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "16c3e70d474d4aa9837d1c505f3a11ad"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "1085240a0829eabccc537aafef2cc37f"
server_key_iv: "1ce201c69c4e4ef6ae845749aad0f604"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.727753 24217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "16c3e70d474d4aa9837d1c505f3a11ad"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "1085240a0829eabccc537aafef2cc37f"
server_key_iv: "1ce201c69c4e4ef6ae845749aad0f604"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.729364 24217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 20:56:39.730195 24236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:39.730412 24217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:39.730523 24217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "16c3e70d474d4aa9837d1c505f3a11ad"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "1085240a0829eabccc537aafef2cc37f"
server_key_iv: "1ce201c69c4e4ef6ae845749aad0f604"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.730608 24217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:39.746095 24217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:39.746462 24217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:39.746631 24217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:39.750993 24217 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:34745
I20260808 20:56:39.751029 24308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:34745 every 8 connection(s)
I20260808 20:56:39.751475 24217 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:39.752295 24309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:39.753388 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 24217
I20260808 20:56:39.753523 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:39.753751 16290 external_mini_cluster.cc:1428] Setting key 3aaf0e202203c096e6795085c506e955
I20260808 20:56:39.755764 24309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad: Bootstrap starting.
I20260808 20:56:39.756517 24309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:39.756805 24309 log.cc:824] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:39.757530 24309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad: No bootstrap required, opened a new log
I20260808 20:56:39.759300 24315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.754326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55032 (local address 127.15.232.190:34745)
0808 20:56:39.754553 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.754561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.754586 (+ 25us) server_negotiation.cc:407] Connection header received
0808 20:56:39.755016 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.755022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.755083 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.755230 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:39.755421 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.756395 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.757304 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.757760 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.757903 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.758659 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:39.758663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:39.758663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:39.758824 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:39.758859 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:39.758869 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:39.758965 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:39.759057 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 20:56:39.759141 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260808 20:56:39.759735 24309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c3e70d474d4aa9837d1c505f3a11ad" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34745 } }
I20260808 20:56:39.759938 24309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:39.759989 24309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c3e70d474d4aa9837d1c505f3a11ad, State: Initialized, Role: FOLLOWER
I20260808 20:56:39.760114 24309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c3e70d474d4aa9837d1c505f3a11ad" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34745 } }
I20260808 20:56:39.760201 24309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:39.760244 24309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:39.760293 24309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:39.760397 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 20:56:39.760922 24309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c3e70d474d4aa9837d1c505f3a11ad" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34745 } }
I20260808 20:56:39.761046 24309 leader_election.cc:302] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16c3e70d474d4aa9837d1c505f3a11ad; no voters:
I20260808 20:56:39.761257 24309 leader_election.cc:288] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:39.761302 24318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:39.761427 24318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [term 1 LEADER]: Becoming Leader. State: Replica: 16c3e70d474d4aa9837d1c505f3a11ad, State: Running, Role: LEADER
I20260808 20:56:39.761511 24318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c3e70d474d4aa9837d1c505f3a11ad" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34745 } }
I20260808 20:56:39.761554 24309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:39.761855 24318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16c3e70d474d4aa9837d1c505f3a11ad. Latest consensus state: current_term: 1 leader_uuid: "16c3e70d474d4aa9837d1c505f3a11ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c3e70d474d4aa9837d1c505f3a11ad" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34745 } } }
I20260808 20:56:39.761929 24318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:39.761886 24322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16c3e70d474d4aa9837d1c505f3a11ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c3e70d474d4aa9837d1c505f3a11ad" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34745 } } }
I20260808 20:56:39.761978 24322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad [sys.catalog]: This master's current role is: LEADER
W20260808 20:56:39.763595 24346 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 20:56:39.763681 24346 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 20:56:39.763731 24326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:39.763885 24326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:39.765445 24326 catalog_manager.cc:1347] Generated new cluster ID: bb3d6595075740c2b6c6f7c09eefa7cd
I20260808 20:56:39.765511 24326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:39.783027 24326 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:39.783752 24326 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:39.787263 24326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16c3e70d474d4aa9837d1c505f3a11ad: Generated new TSK 0
I20260808 20:56:39.787505 24326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 20:56:39.799222 24315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.795330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.15.232.190:34745)
0808 20:56:39.795397 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.795399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.795548 (+ 149us) server_negotiation.cc:407] Connection header received
0808 20:56:39.795636 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.795638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.795651 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.795727 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0808 20:56:39.795904 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.796709 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.797495 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.797814 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.797886 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.798031 (+ 145us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 20:56:39.799008 (+ 977us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 20:56:39.799147 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55046: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 20:56:39.799290 24315 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55046: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 20:56:39.801270 24358 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:39.795215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:34745 (local address 127.0.0.1:55046)
0808 20:56:39.795455 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:39.795483 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:39.795557 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:39.795763 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:39.795764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:39.795766 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 20:56:39.795825 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:39.795826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.796795 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.796796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:39.797400 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:39.797402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.797897 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.797897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:39.797939 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.797944 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 20:56:39.799118 (+ 1174us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 20:56:39.801170 (+ 2052us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.232.190:34745: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37253/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":1843,"thread_start_us":131,"threads_started":1}
I20260808 20:56:39.801641 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 24217
E20260808 20:56:39.815121 24359 webserver.cc:533] Webserver: sslize failed:
E20260808 20:56:39.815239 24359 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T20:56:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T20:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:39Z Disabled control of system clock
I20260808 20:56:39.861195 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:42385
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:42845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:42385
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/testchainca.cert with env {}
W20260808 20:56:39.945659 24363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:39.945901 24363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:39.945926 24363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:39.947088 24363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:39.947126 24363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:39.947139 24363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:39.947152 24363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:39.947165 24363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 20:56:39.947177 24363 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 20:56:39.947188 24363 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 20:56:39.948691 24363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:42845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:42385
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:42385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:39.948905 24363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:39.949070 24363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:39.954512 24371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:39.954512 24375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:39.954545 24372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:39.954543 24363 server_base.cc:1034] running on GCE node
I20260808 20:56:39.955107 24363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:39.955412 24363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:39.956630 24363 hybrid_clock.cc:648] HybridClock initialized: now 1786222599956616 us; error 32 us; skew 500 ppm
I20260808 20:56:39.958039 24363 webserver.cc:460] Webserver started at http://127.15.232.190:33931/ using document root <none> and password file <none>
I20260808 20:56:39.958302 24363 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:39.958379 24363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:39.958540 24363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:39.959802 24363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "57f377329c104ee5bf35ca03fd8e261a"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "28015a428fcdfb24d7e27508628ca504"
server_key_iv: "ebfd9c96593b387410c7c5802278d5f8"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.960206 24363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "57f377329c104ee5bf35ca03fd8e261a"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "28015a428fcdfb24d7e27508628ca504"
server_key_iv: "ebfd9c96593b387410c7c5802278d5f8"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.961804 24363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:39.962762 24383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:39.963091 24363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:39.963215 24363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "57f377329c104ee5bf35ca03fd8e261a"
format_stamp: "Formatted at 2026-08-08 20:56:39 on dist-test-slave-c12x"
server_key: "28015a428fcdfb24d7e27508628ca504"
server_key_iv: "ebfd9c96593b387410c7c5802278d5f8"
server_key_version: "encryptionkey@0"
I20260808 20:56:39.963301 24363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:39.978706 24363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:39.979074 24363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:39.979239 24363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:39.983664 24363 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:42385
I20260808 20:56:39.983722 24456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:42385 every 8 connection(s)
I20260808 20:56:39.984103 24363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:39.985131 24457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:39.986495 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 24363
I20260808 20:56:39.986615 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:39.986835 16290 external_mini_cluster.cc:1428] Setting key 022b7068a5e7d10efdc85f2248a68f2e
I20260808 20:56:39.988178 24457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a: Bootstrap starting.
I20260808 20:56:39.989037 24457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:39.989387 24457 log.cc:824] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:39.990260 24457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a: No bootstrap required, opened a new log
I20260808 20:56:39.992571 24457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f377329c104ee5bf35ca03fd8e261a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42385 } }
I20260808 20:56:39.992753 24457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:39.992796 24457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f377329c104ee5bf35ca03fd8e261a, State: Initialized, Role: FOLLOWER
I20260808 20:56:39.992949 24457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f377329c104ee5bf35ca03fd8e261a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42385 } }
I20260808 20:56:39.993050 24457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:39.993086 24457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:39.993151 24457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:39.993292 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:39.987385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.15.232.190:42385)
0808 20:56:39.987638 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:39.987647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:39.987685 (+ 38us) server_negotiation.cc:407] Connection header received
0808 20:56:39.988240 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:39.988250 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:39.988339 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:39.988497 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:39.988681 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.989994 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.990860 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:39.991502 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:39.991637 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:39.992366 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:39.992370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:39.992370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:56:39.992582 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:39.992627 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:56:39.992642 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:39.992715 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:39.992935 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 20:56:39.993019 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":124,"threads_started":1}
I20260808 20:56:39.993912 24457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f377329c104ee5bf35ca03fd8e261a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42385 } }
I20260808 20:56:39.994040 24457 leader_election.cc:302] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57f377329c104ee5bf35ca03fd8e261a; no voters:
I20260808 20:56:39.994203 24457 leader_election.cc:288] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:39.994238 24463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:39.994398 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 20:56:39.994413 24463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [term 1 LEADER]: Becoming Leader. State: Replica: 57f377329c104ee5bf35ca03fd8e261a, State: Running, Role: LEADER
I20260808 20:56:39.994460 24457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:39.994560 24463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f377329c104ee5bf35ca03fd8e261a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42385 } }
I20260808 20:56:39.994886 24463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57f377329c104ee5bf35ca03fd8e261a. Latest consensus state: current_term: 1 leader_uuid: "57f377329c104ee5bf35ca03fd8e261a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f377329c104ee5bf35ca03fd8e261a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42385 } } }
I20260808 20:56:39.994971 24463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:39.994900 24464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57f377329c104ee5bf35ca03fd8e261a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f377329c104ee5bf35ca03fd8e261a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42385 } } }
I20260808 20:56:39.995010 24464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:39.995312 24472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:39.995882 24472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:39.997462 24472 catalog_manager.cc:1347] Generated new cluster ID: 258be8a0640f4d2fbe00a58c5b5b0b75
I20260808 20:56:39.997620 24472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:40.007683 24472 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:40.008270 24472 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:40.018788 24472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57f377329c104ee5bf35ca03fd8e261a: Generated new TSK 0
I20260808 20:56:40.019039 24472 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 20:56:40.904135 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:40.027144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.15.232.190:42385)
0808 20:56:40.027194 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:40.027195 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:40.027207 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:56:40.027312 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:40.027312 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:40.027323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:40.027391 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0808 20:56:40.027585 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:40.028093 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:40.029242 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:40.029452 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:40.029508 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:40.029762 (+ 254us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 20:56:40.903805 (+874043us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 20:56:40.904048 (+ 243us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34024: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 20:56:40.904219 24497 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:40.027003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:42385 (local address 127.0.0.1:34024)
0808 20:56:40.027158 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:40.027169 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:40.027226 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:40.027417 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:40.027418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:40.027420 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 20:56:40.027522 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:40.027525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:40.028185 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:40.028186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:40.029149 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:40.029153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:40.029545 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:40.029546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:40.029611 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:40.029617 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 20:56:40.903980 (+874363us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 20:56:40.904108 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.232.190:42385: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
W20260808 20:56:40.904234 24460 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34024: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260808 20:56:40.904551 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 24363
2026-08-08T20:56:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1078 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (98 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:41 dist-test-slave-c12x krb5kdc[24511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:41 dist-test-slave-c12x krb5kdc[24511](info): set up 2 sockets
Aug 08 20:56:41 dist-test-slave-c12x krb5kdc[24511](info): commencing operation
krb5kdc: starting...
W20260808 20:56:42.325774 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:42 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:42.462610 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37985
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:37985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:42.549068 24530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:42.549358 24530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:42.549399 24530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:42.550618 24530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:42.550693 24530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:42.550717 24530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:42.550746 24530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:42.550778 24530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:42.552407 24530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:37985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:42.552709 24530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:42.552963 24530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:42.558571 24540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:42.558627 24543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:42.558729 24539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:42.559034 24530 server_base.cc:1034] running on GCE node
I20260808 20:56:42.559337 24530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:42.559692 24530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:42.560850 24530 hybrid_clock.cc:648] HybridClock initialized: now 1786222602560827 us; error 42 us; skew 500 ppm
Aug 08 20:56:42 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222602, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:42.566712 24530 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:42.567276 24530 webserver.cc:460] Webserver started at http://127.15.232.190:41823/ using document root <none> and password file <none>
I20260808 20:56:42.567567 24530 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:42.567619 24530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:42.567795 24530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:42.568994 24530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "76cdc08f4ac14e6abda2fc0b687102fd"
format_stamp: "Formatted at 2026-08-08 20:56:42 on dist-test-slave-c12x"
server_key: "548335fc0527a6f821dce0f4d637bc6a"
server_key_iv: "132c07ff0e5da3bf50509dd4bee8aa36"
server_key_version: "encryptionkey@0"
I20260808 20:56:42.569402 24530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "76cdc08f4ac14e6abda2fc0b687102fd"
format_stamp: "Formatted at 2026-08-08 20:56:42 on dist-test-slave-c12x"
server_key: "548335fc0527a6f821dce0f4d637bc6a"
server_key_iv: "132c07ff0e5da3bf50509dd4bee8aa36"
server_key_version: "encryptionkey@0"
I20260808 20:56:42.570904 24530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:42.571799 24554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:42.572026 24530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:42.572184 24530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "76cdc08f4ac14e6abda2fc0b687102fd"
format_stamp: "Formatted at 2026-08-08 20:56:42 on dist-test-slave-c12x"
server_key: "548335fc0527a6f821dce0f4d637bc6a"
server_key_iv: "132c07ff0e5da3bf50509dd4bee8aa36"
server_key_version: "encryptionkey@0"
I20260808 20:56:42.572281 24530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:42.583757 24530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:42.586236 24530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:42.586421 24530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:42.590929 24530 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:37985
I20260808 20:56:42.590962 24633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:37985 every 8 connection(s)
I20260808 20:56:42.591334 24530 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:42.592303 24634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:42.594645 24634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd: Bootstrap starting.
I20260808 20:56:42.595325 24634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:42.595676 24634 log.cc:824] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:42.596375 24634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd: No bootstrap required, opened a new log
I20260808 20:56:42.598232 24634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cdc08f4ac14e6abda2fc0b687102fd" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37985 } }
I20260808 20:56:42.598389 24634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:42.598413 24634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76cdc08f4ac14e6abda2fc0b687102fd, State: Initialized, Role: FOLLOWER
I20260808 20:56:42.598583 24634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cdc08f4ac14e6abda2fc0b687102fd" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37985 } }
I20260808 20:56:42.598734 24634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:42.598788 24634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:42.598839 24634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:42.598919 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 24530
I20260808 20:56:42.599062 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:42.599254 16290 external_mini_cluster.cc:1428] Setting key 7ea91fd62f0d8cd20bf6cadefc1d9640
I20260808 20:56:42.599720 24634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cdc08f4ac14e6abda2fc0b687102fd" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37985 } }
I20260808 20:56:42.599948 24634 leader_election.cc:302] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76cdc08f4ac14e6abda2fc0b687102fd; no voters:
I20260808 20:56:42.600178 24634 leader_election.cc:288] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:42.600416 24640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:42.600564 24634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:42.600589 24640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [term 1 LEADER]: Becoming Leader. State: Replica: 76cdc08f4ac14e6abda2fc0b687102fd, State: Running, Role: LEADER
I20260808 20:56:42.601055 24640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cdc08f4ac14e6abda2fc0b687102fd" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37985 } }
I20260808 20:56:42.601496 24644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76cdc08f4ac14e6abda2fc0b687102fd. Latest consensus state: current_term: 1 leader_uuid: "76cdc08f4ac14e6abda2fc0b687102fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cdc08f4ac14e6abda2fc0b687102fd" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37985 } } }
I20260808 20:56:42.601604 24644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:42.601967 24641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76cdc08f4ac14e6abda2fc0b687102fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76cdc08f4ac14e6abda2fc0b687102fd" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37985 } } }
I20260808 20:56:42.602059 24641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:42.602207 24652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:42.602753 24652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:42.604733 24652 catalog_manager.cc:1347] Generated new cluster ID: 40c1ca512c0641dfb848c2e17d735529
I20260808 20:56:42.604794 24652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 20:56:42 dist-test-slave-c12x krb5kdc[24511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:42.612224 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:42.599855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38316 (local address 127.15.232.190:37985)
0808 20:56:42.600058 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:42.600069 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:42.600111 (+ 42us) server_negotiation.cc:407] Connection header received
0808 20:56:42.600242 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:42.600250 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:42.600316 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:42.601333 (+ 1017us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:42.601438 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.603445 (+ 2007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.604677 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.605303 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.605419 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:42.608426 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:42.608431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:42.608431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:42.608444 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:42.610473 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:42.610910 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:42.610913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:42.610914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:42.611029 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:42.611253 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:42.611253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:42.611254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:42.611592 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:42.611733 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:42.611865 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 20:56:42.612003 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":107,"threads_started":1}
I20260808 20:56:42.639308 24652 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:42.639914 24652 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:42.646835 24652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76cdc08f4ac14e6abda2fc0b687102fd: Generated new TSK 0
I20260808 20:56:42.647100 24652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:42.682631 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37985
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:42.786661 24670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:42.786877 24670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:42.786902 24670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:42.788134 24670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:42.788175 24670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:42.788203 24670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:42.789737 24670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:42.790002 24670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:42.790239 24670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:42.795832 24678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:42.795807 24683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:42.795850 24670 server_base.cc:1034] running on GCE node
W20260808 20:56:42.795820 24679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:42.796213 24670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:42.796502 24670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:42.797703 24670 hybrid_clock.cc:648] HybridClock initialized: now 1786222602797684 us; error 39 us; skew 500 ppm
Aug 08 20:56:42 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222602, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:42.800351 24670 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:42.800910 24670 webserver.cc:460] Webserver started at http://127.15.232.129:33715/ using document root <none> and password file <none>
I20260808 20:56:42.801162 24670 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:42.801235 24670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:42.801375 24670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:42.802654 24670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "0d5a4e0518124061b71603157a78b59e"
format_stamp: "Formatted at 2026-08-08 20:56:42 on dist-test-slave-c12x"
server_key: "7beb28b6446d550bee27a9026d5ee77b"
server_key_iv: "4e34c8e53a7774e53938c0a14d3e1c3e"
server_key_version: "encryptionkey@0"
I20260808 20:56:42.803047 24670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "0d5a4e0518124061b71603157a78b59e"
format_stamp: "Formatted at 2026-08-08 20:56:42 on dist-test-slave-c12x"
server_key: "7beb28b6446d550bee27a9026d5ee77b"
server_key_iv: "4e34c8e53a7774e53938c0a14d3e1c3e"
server_key_version: "encryptionkey@0"
I20260808 20:56:42.804625 24670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:42.805477 24691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:42.805720 24670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:42.805846 24670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "0d5a4e0518124061b71603157a78b59e"
format_stamp: "Formatted at 2026-08-08 20:56:42 on dist-test-slave-c12x"
server_key: "7beb28b6446d550bee27a9026d5ee77b"
server_key_iv: "4e34c8e53a7774e53938c0a14d3e1c3e"
server_key_version: "encryptionkey@0"
I20260808 20:56:42.805940 24670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:42.815606 24670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:42.817523 24670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:42.817693 24670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:42.817951 24670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:42.818275 24670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:42.818334 24670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:42.818370 24670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:42.818392 24670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:42.825012 24670 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:39247
I20260808 20:56:42.825065 24854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:39247 every 8 connection(s)
I20260808 20:56:42.825431 24670 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:42.829541 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 24670
I20260808 20:56:42.829656 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:42.829830 16290 external_mini_cluster.cc:1428] Setting key 51c1029c6e477f21c40d83284774cd51
Aug 08 20:56:42 dist-test-slave-c12x krb5kdc[24511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222602, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:42.836393 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:42.826234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:60549 (local address 127.15.232.190:37985)
0808 20:56:42.826307 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:42.826309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:42.826546 (+ 237us) server_negotiation.cc:407] Connection header received
0808 20:56:42.827542 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:42.827543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:42.827556 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:42.827610 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:42.828275 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.828794 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.830142 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.830445 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.830516 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:42.832928 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:42.832929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:42.832929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:42.832936 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:42.834681 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:42.835135 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:42.835136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:42.835137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:42.835207 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:42.835565 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:42.835566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:42.835566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:42.835717 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:42.835795 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:42.836151 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 20:56:42.836252 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 20:56:42.836413 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:42.826209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37985 (local address 127.15.232.129:60549)
0808 20:56:42.826397 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:42.826433 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:42.827389 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:42.827709 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:42.827713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:42.827968 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:42.828177 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:42.828181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.828896 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.828897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:42.830047 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:42.830050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.830539 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.830540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:42.830685 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:42.831220 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:42.831229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:42.832812 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:42.834795 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:42.834802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:42.834804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:42.835043 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:42.835306 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:42.835307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:42.835307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:42.835428 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:42.835824 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:42.835831 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:42.835982 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 20:56:42.836073 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 20:56:42.836219 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260808 20:56:42.837183 24855 heartbeater.cc:347] Connected to a master server at 127.15.232.190:37985
I20260808 20:56:42.837301 24855 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:42.837515 24855 heartbeater.cc:511] Master 127.15.232.190:37985 requested a full tablet report, sending...
I20260808 20:56:42.838068 24579 ts_manager.cc:194] Registered new tserver with Master: 0d5a4e0518124061b71603157a78b59e (127.15.232.129:39247)
I20260808 20:56:42.838671 24579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:60549
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:42.858875 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37985
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:42.947824 24863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:42.948115 24863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:42.948163 24863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:42.949416 24863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:42.949499 24863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:42.949548 24863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:42.951153 24863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:42.951478 24863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:42.951736 24863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:42.957160 24868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:42.957170 24873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:42.957283 24871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:42.957342 24863 server_base.cc:1034] running on GCE node
I20260808 20:56:42.957655 24863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:42.958029 24863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:42.959215 24863 hybrid_clock.cc:648] HybridClock initialized: now 1786222602959202 us; error 31 us; skew 500 ppm
Aug 08 20:56:42 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222602, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:42.962163 24863 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:42.962703 24863 webserver.cc:460] Webserver started at http://127.15.232.130:40795/ using document root <none> and password file <none>
I20260808 20:56:42.962966 24863 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:42.963043 24863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:42.963208 24863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:42.964370 24863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "a6136abcea79457787353c4de0550cb4"
format_stamp: "Formatted at 2026-08-08 20:56:42 on dist-test-slave-c12x"
server_key: "6d27ad5b7103c20e28650edf510b2215"
server_key_iv: "2e471b005dea971c22c7df1bb0425ed0"
server_key_version: "encryptionkey@0"
I20260808 20:56:42.964742 24863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "a6136abcea79457787353c4de0550cb4"
format_stamp: "Formatted at 2026-08-08 20:56:42 on dist-test-slave-c12x"
server_key: "6d27ad5b7103c20e28650edf510b2215"
server_key_iv: "2e471b005dea971c22c7df1bb0425ed0"
server_key_version: "encryptionkey@0"
I20260808 20:56:42.966234 24863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 20:56:42.967072 24882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:42.967257 24863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:42.967362 24863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "a6136abcea79457787353c4de0550cb4"
format_stamp: "Formatted at 2026-08-08 20:56:42 on dist-test-slave-c12x"
server_key: "6d27ad5b7103c20e28650edf510b2215"
server_key_iv: "2e471b005dea971c22c7df1bb0425ed0"
server_key_version: "encryptionkey@0"
I20260808 20:56:42.967473 24863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:42.982738 24863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:42.985340 24863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:42.985515 24863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:42.985797 24863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:42.986151 24863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:42.986210 24863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:42.986246 24863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:42.986274 24863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:42.993274 24863 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:42781
I20260808 20:56:42.993321 25045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:42781 every 8 connection(s)
I20260808 20:56:42.993677 24863 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:42.995328 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 24863
I20260808 20:56:42.995471 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:42.995661 16290 external_mini_cluster.cc:1428] Setting key 470d87715b29e824024f24f57b21083f
Aug 08 20:56:42 dist-test-slave-c12x krb5kdc[24511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222602, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:43.005252 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:42.994495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:35325 (local address 127.15.232.190:37985)
0808 20:56:42.994579 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:42.994581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:42.994784 (+ 203us) server_negotiation.cc:407] Connection header received
0808 20:56:42.995899 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:42.995901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:42.995914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:42.995971 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:42.996718 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.997460 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.998810 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.999004 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.999068 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:43.001968 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:43.001970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:43.001970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:43.001979 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:43.003624 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.004174 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.004175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.004175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.004232 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.004540 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.004541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.004541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.004679 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:43.004732 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:43.005104 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 20:56:43.005181 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 20:56:43.005591 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:42.994432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37985 (local address 127.15.232.130:35325)
0808 20:56:42.994655 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:42.994696 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:42.995793 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:42.996048 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:42.996053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:42.996411 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:42.996618 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:42.996623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.997556 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.997557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:42.998711 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:42.998715 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:42.999098 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:42.999099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:42.999263 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:42.999902 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:42.999911 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:43.001828 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:43.003708 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:43.003718 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:43.003721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:43.004086 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:43.004302 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:43.004303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:43.004304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:43.004462 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:43.004748 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:43.004755 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:43.004947 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 20:56:43.005086 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 20:56:43.005289 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
I20260808 20:56:43.006222 25046 heartbeater.cc:347] Connected to a master server at 127.15.232.190:37985
I20260808 20:56:43.006342 25046 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:43.006520 25046 heartbeater.cc:511] Master 127.15.232.190:37985 requested a full tablet report, sending...
I20260808 20:56:43.006899 24579 ts_manager.cc:194] Registered new tserver with Master: a6136abcea79457787353c4de0550cb4 (127.15.232.130:42781)
I20260808 20:56:43.007330 24579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:35325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:43.026875 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37985
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:33631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:43.112641 25053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:43.112859 25053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:43.112891 25053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:43.114075 25053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:43.114118 25053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:43.114145 25053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:43.115991 25053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:33631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:43.116259 25053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:43.116540 25053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:43.121798 25062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:43.121816 25063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:43.121821 25065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:43.122136 25053 server_base.cc:1034] running on GCE node
I20260808 20:56:43.122383 25053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:43.122640 25053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:43.123809 25053 hybrid_clock.cc:648] HybridClock initialized: now 1786222603123802 us; error 43 us; skew 500 ppm
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:43.126506 25053 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:43.126955 25053 webserver.cc:460] Webserver started at http://127.15.232.131:41403/ using document root <none> and password file <none>
I20260808 20:56:43.127221 25053 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:43.127292 25053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:43.127486 25053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:43.128582 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "02e2b2d06f07444692e1876c14d0c648"
format_stamp: "Formatted at 2026-08-08 20:56:43 on dist-test-slave-c12x"
server_key: "e16bbf5abc4056e5d74cce88626624b4"
server_key_iv: "b9a0201d7924ce66291291386e833172"
server_key_version: "encryptionkey@0"
I20260808 20:56:43.129032 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "02e2b2d06f07444692e1876c14d0c648"
format_stamp: "Formatted at 2026-08-08 20:56:43 on dist-test-slave-c12x"
server_key: "e16bbf5abc4056e5d74cce88626624b4"
server_key_iv: "b9a0201d7924ce66291291386e833172"
server_key_version: "encryptionkey@0"
I20260808 20:56:43.130477 25053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:43.131207 25073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:43.131382 25053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 20:56:43.131511 25053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "02e2b2d06f07444692e1876c14d0c648"
format_stamp: "Formatted at 2026-08-08 20:56:43 on dist-test-slave-c12x"
server_key: "e16bbf5abc4056e5d74cce88626624b4"
server_key_iv: "b9a0201d7924ce66291291386e833172"
server_key_version: "encryptionkey@0"
I20260808 20:56:43.131603 25053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:43.145723 25053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:43.148200 25053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:43.148375 25053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:43.148644 25053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:43.148995 25053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:43.149057 25053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:43.149113 25053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:43.149161 25053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:43.155246 25053 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:34725
I20260808 20:56:43.155328 25249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:34725 every 8 connection(s)
I20260808 20:56:43.155854 25053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:43.163213 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 25053
I20260808 20:56:43.163319 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:43.163554 16290 external_mini_cluster.cc:1428] Setting key cb419570966a7ccffd66e4a2484c0e9e
I20260808 20:56:43.166344 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:43.156731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:52541 (local address 127.15.232.190:37985)
0808 20:56:43.156787 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:43.156790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:43.156972 (+ 182us) server_negotiation.cc:407] Connection header received
0808 20:56:43.157957 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:43.157959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:43.157970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:43.158048 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:43.158697 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.159205 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.160282 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.160468 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.160532 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:43.162780 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:43.162782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:43.162782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:43.162790 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:43.164758 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.165235 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.165236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.165236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.165311 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.165596 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.165597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.165598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.165742 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:43.165832 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:43.166141 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 20:56:43.166245 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:56:43.166625 25252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:43.156673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37985 (local address 127.15.232.131:52541)
0808 20:56:43.156841 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:43.156885 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:43.157850 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:43.158121 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:43.158124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:43.158398 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:43.158613 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:43.158617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.159284 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.159285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:43.160195 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:43.160197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.160544 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.160544 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:43.160668 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:43.161246 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:43.161255 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:43.162674 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:43.164834 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:43.164842 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:43.164845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:43.165128 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:43.165390 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:43.165391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:43.165392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:43.165515 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:43.165821 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:43.165828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:43.165982 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 20:56:43.166103 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 20:56:43.166287 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260808 20:56:43.167248 25250 heartbeater.cc:347] Connected to a master server at 127.15.232.190:37985
I20260808 20:56:43.167379 25250 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:43.167637 25250 heartbeater.cc:511] Master 127.15.232.190:37985 requested a full tablet report, sending...
I20260808 20:56:43.168064 24579 ts_manager.cc:194] Registered new tserver with Master: 02e2b2d06f07444692e1876c14d0c648 (127.15.232.131:34725)
I20260808 20:56:43.168468 24579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:52541
I20260808 20:56:43.175869 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/test-admin.keytab.
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:43.194419 16290 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 20:56:43.196208 25257 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:43.195296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57834 (local address 127.15.232.129:39247)
0808 20:56:43.195508 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:43.195513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:43.195523 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:43.195558 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:43.195560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:43.195573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:43.195645 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:43.196100 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57834: BlockingRecv error: recv got EOF from 127.0.0.1:57834 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":116,"threads_started":1}
W20260808 20:56:43.196352 16290 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5cc' of type FILE
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:43.197417 16290 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:43.198395 16290 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:43.207391 25257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:43.199069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57846 (local address 127.15.232.129:39247)
0808 20:56:43.199118 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:43.199120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:43.199189 (+ 69us) server_negotiation.cc:407] Connection header received
0808 20:56:43.199230 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:43.199231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:43.199242 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:43.199297 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:43.199772 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.200644 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.201451 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.201827 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.201887 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:43.204394 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:43.204403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:43.204403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:43.204411 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:43.206177 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.206600 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.206604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.206604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.206663 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.206891 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.206892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.206892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.207038 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:43.207156 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:43.207241 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 20:56:43.207309 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 20:56:43.209594 25257 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:43.208876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57856 (local address 127.15.232.129:39247)
0808 20:56:43.208935 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:43.208937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:43.209010 (+ 73us) server_negotiation.cc:407] Connection header received
0808 20:56:43.209117 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:43.209118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:43.209130 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:43.209200 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:43.209497 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57856: BlockingRecv error: recv got EOF from 127.0.0.1:57856 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 20:56:43.209894 16290 init.cc:244] Error while opening credential cache '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5cc)
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:43.211004 16290 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:43.212025 16290 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:43.219604 25257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:43.212706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57870 (local address 127.15.232.129:39247)
0808 20:56:43.212788 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:43.212790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:43.212864 (+ 74us) server_negotiation.cc:407] Connection header received
0808 20:56:43.212937 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:43.212937 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:43.212947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:43.212998 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:43.213392 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.213912 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.214644 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.214860 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.214947 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:43.217121 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:43.217122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:43.217123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:43.217131 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:43.218451 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.218803 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.218805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.218805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.218866 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.219075 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.219076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.219076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.219181 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:43.219288 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:43.219391 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 20:56:43.219499 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 20:56:43.221267 25257 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:43.220642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.15.232.129:39247)
0808 20:56:43.220722 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:43.220724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:43.220782 (+ 58us) server_negotiation.cc:407] Connection header received
0808 20:56:43.220854 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:43.220856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:43.220865 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:43.220913 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:43.221220 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57874: BlockingRecv error: recv got EOF from 127.0.0.1:57874 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 20:56:43.221437 16290 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786222573557010-16290-0/krb5kdc/krb5cc' of type FILE
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:43.222436 16290 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:43.223281 16290 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[24511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:56:43.231158 25257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:43.223892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57882 (local address 127.15.232.129:39247)
0808 20:56:43.223951 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:43.223953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:43.224008 (+ 55us) server_negotiation.cc:407] Connection header received
0808 20:56:43.224090 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:43.224091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:43.224100 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:43.224174 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:43.224569 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.225037 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.225723 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:43.225925 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:43.226002 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:43.228504 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:43.228506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:43.228506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:43.228512 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:43.229957 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.230306 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.230307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.230307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.230368 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:43.230588 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:43.230589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:43.230589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:43.230700 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:43.230782 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:43.230960 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 20:56:43.231025 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 20:56:43.232024 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 24670
I20260808 20:56:43.247828 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 24863
I20260808 20:56:43.265465 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 25053
I20260808 20:56:43.287567 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 24530
2026-08-08T20:56:43Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2191 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[25278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[25278](info): set up 2 sockets
Aug 08 20:56:43 dist-test-slave-c12x krb5kdc[25278](info): commencing operation
krb5kdc: starting...
W20260808 20:56:44.509521 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:44 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.190.keytab.
Entry for principal oryx/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.190.keytab.
I20260808 20:56:44.642459 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33823
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.190.keytab
--principal=oryx/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:33823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:44.726851 25295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:44.727072 25295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:44.727095 25295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:44.728322 25295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:44.728375 25295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:44.728390 25295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:44.728404 25295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:44.728463 25295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:44.729961 25295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:33823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.190.keytab
--principal=oryx/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:44.730232 25295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:44.730485 25295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:44.736198 25304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:44.736192 25303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:44.736192 25307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:44.736644 25295 server_base.cc:1034] running on GCE node
I20260808 20:56:44.736884 25295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:44.737160 25295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:44.738303 25295 hybrid_clock.cc:648] HybridClock initialized: now 1786222604738282 us; error 42 us; skew 500 ppm
Aug 08 20:56:44 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222604, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:44.740850 25295 init.cc:377] Logged in from keytab as oryx/127.15.232.190@KRBTEST.COM (short username oryx)
I20260808 20:56:44.741344 25295 webserver.cc:460] Webserver started at http://127.15.232.190:43735/ using document root <none> and password file <none>
I20260808 20:56:44.741545 25295 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:44.741588 25295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:44.741684 25295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:44.742754 25295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "ea87f99a39bd455392ccdd6f0fe2909e"
format_stamp: "Formatted at 2026-08-08 20:56:44 on dist-test-slave-c12x"
server_key: "5721de09666c4467ea52bffc60a9cf9f"
server_key_iv: "749f3578b6b9aee6043c3f3db4d53a30"
server_key_version: "encryptionkey@0"
I20260808 20:56:44.743146 25295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "ea87f99a39bd455392ccdd6f0fe2909e"
format_stamp: "Formatted at 2026-08-08 20:56:44 on dist-test-slave-c12x"
server_key: "5721de09666c4467ea52bffc60a9cf9f"
server_key_iv: "749f3578b6b9aee6043c3f3db4d53a30"
server_key_version: "encryptionkey@0"
I20260808 20:56:44.744634 25295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:44.745378 25315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:44.745582 25295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:44.745700 25295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "ea87f99a39bd455392ccdd6f0fe2909e"
format_stamp: "Formatted at 2026-08-08 20:56:44 on dist-test-slave-c12x"
server_key: "5721de09666c4467ea52bffc60a9cf9f"
server_key_iv: "749f3578b6b9aee6043c3f3db4d53a30"
server_key_version: "encryptionkey@0"
I20260808 20:56:44.745779 25295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:44.763772 25295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:44.766196 25295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:44.766376 25295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:44.770655 25295 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:33823
I20260808 20:56:44.770741 25394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:33823 every 8 connection(s)
I20260808 20:56:44.771116 25295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:44.772251 25395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:44.774549 25395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: Bootstrap starting.
I20260808 20:56:44.775121 25395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:44.775354 25395 log.cc:824] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:44.776165 25395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: No bootstrap required, opened a new log
I20260808 20:56:44.777724 25395 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } }
I20260808 20:56:44.777869 25395 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:44.777894 25395 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea87f99a39bd455392ccdd6f0fe2909e, State: Initialized, Role: FOLLOWER
I20260808 20:56:44.778047 25395 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } }
I20260808 20:56:44.778131 25395 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:44.778153 25395 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:44.778182 25395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:44.778935 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 25295
I20260808 20:56:44.779070 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:44.779147 25395 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } }
I20260808 20:56:44.779268 25395 leader_election.cc:302] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea87f99a39bd455392ccdd6f0fe2909e; no voters:
I20260808 20:56:44.779291 16290 external_mini_cluster.cc:1428] Setting key 7d0bf4234c466e4dc07895d64a83e5b5
I20260808 20:56:44.779479 25395 leader_election.cc:288] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:44.779513 25400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:44.779759 25400 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 1 LEADER]: Becoming Leader. State: Replica: ea87f99a39bd455392ccdd6f0fe2909e, State: Running, Role: LEADER
I20260808 20:56:44.779837 25395 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:44.779904 25400 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } }
I20260808 20:56:44.780411 25400 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea87f99a39bd455392ccdd6f0fe2909e. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } }
I20260808 20:56:44.780673 25400 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:44.780599 25401 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } }
I20260808 20:56:44.780956 25401 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:44.781186 25415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:44.781792 25415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:44.783535 25415 catalog_manager.cc:1347] Generated new cluster ID: 52d888ded2cb4dba8eba7e35c91af6e6
I20260808 20:56:44.783593 25415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 20:56:44 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:44.791325 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:44.780126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.15.232.190:33823)
0808 20:56:44.780364 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:44.780374 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:44.780412 (+ 38us) server_negotiation.cc:407] Connection header received
0808 20:56:44.780583 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:44.780592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:44.780656 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:44.781534 (+ 878us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:44.781755 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:44.783228 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:44.784161 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:44.784603 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:44.784727 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:44.787277 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:44.787281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:44.787282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:44.787295 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:44.789590 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:44.789994 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:44.789997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:44.789997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:44.790117 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:44.790342 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:44.790343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:44.790344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:44.790626 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:44.790801 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:44.790922 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 20:56:44.791075 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":104,"threads_started":1}
I20260808 20:56:44.812932 25415 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:44.813733 25415 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:44.823012 25415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: Generated new TSK 0
I20260808 20:56:44.823254 25415 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab.
Entry for principal oryx/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab.
I20260808 20:56:44.856103 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33823
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab
--principal=oryx/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:44.943768 25431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:44.943991 25431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:44.944015 25431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:44.945212 25431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:44.945259 25431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:44.945286 25431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:44.946800 25431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab
--principal=oryx/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:44.947064 25431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:44.947306 25431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:44.952778 25442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:44.952785 25439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:44.952790 25438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:44.953184 25431 server_base.cc:1034] running on GCE node
I20260808 20:56:44.953377 25431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:44.953584 25431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:44.954710 25431 hybrid_clock.cc:648] HybridClock initialized: now 1786222604954696 us; error 29 us; skew 500 ppm
Aug 08 20:56:44 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222604, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:44.957314 25431 init.cc:377] Logged in from keytab as oryx/127.15.232.129@KRBTEST.COM (short username oryx)
I20260808 20:56:44.957770 25431 webserver.cc:460] Webserver started at http://127.15.232.129:35599/ using document root <none> and password file <none>
I20260808 20:56:44.958026 25431 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:44.958076 25431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:44.958211 25431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:44.959314 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "5288b5f446634ee39afe9dff54506397"
format_stamp: "Formatted at 2026-08-08 20:56:44 on dist-test-slave-c12x"
server_key: "4638de6eac46eeb313c7cb7fdcca1c2f"
server_key_iv: "6d93f2c35bd6210a88511f6f312dea0e"
server_key_version: "encryptionkey@0"
I20260808 20:56:44.959774 25431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "5288b5f446634ee39afe9dff54506397"
format_stamp: "Formatted at 2026-08-08 20:56:44 on dist-test-slave-c12x"
server_key: "4638de6eac46eeb313c7cb7fdcca1c2f"
server_key_iv: "6d93f2c35bd6210a88511f6f312dea0e"
server_key_version: "encryptionkey@0"
I20260808 20:56:44.961187 25431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:44.961977 25456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:44.962178 25431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:44.962287 25431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "5288b5f446634ee39afe9dff54506397"
format_stamp: "Formatted at 2026-08-08 20:56:44 on dist-test-slave-c12x"
server_key: "4638de6eac46eeb313c7cb7fdcca1c2f"
server_key_iv: "6d93f2c35bd6210a88511f6f312dea0e"
server_key_version: "encryptionkey@0"
I20260808 20:56:44.962374 25431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:44.976716 25431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:44.978843 25431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:44.979029 25431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:44.979282 25431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:44.979698 25431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:44.979763 25431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:44.979800 25431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:44.979823 25431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:44.986120 25431 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:33485
I20260808 20:56:44.986207 25628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:33485 every 8 connection(s)
I20260808 20:56:44.986545 25431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:44.993046 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 25431
I20260808 20:56:44.993177 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:44.993394 16290 external_mini_cluster.cc:1428] Setting key 6c12f444866cc49939ede155f6e03605
Aug 08 20:56:44 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222604, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:44.998689 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:44.987365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:49781 (local address 127.15.232.190:33823)
0808 20:56:44.987471 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:44.987474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:44.987759 (+ 285us) server_negotiation.cc:407] Connection header received
0808 20:56:44.988326 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:44.988328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:44.988340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:44.988411 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:44.989857 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:44.990692 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:44.992203 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:44.992413 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:44.992476 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:44.995497 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:44.995499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:44.995499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:44.995508 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:44.997122 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:44.997661 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:44.997662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:44.997662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:44.997718 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:44.998024 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:44.998025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:44.998025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:44.998138 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:44.998192 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:44.998549 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 20:56:44.998623 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 20:56:44.998966 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:44.987332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.129:49781)
0808 20:56:44.987619 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:44.987659 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:44.988208 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:44.988497 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:44.988501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:44.989433 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:44.989721 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:44.989730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:44.990815 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:44.990816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:44.992103 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:44.992106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:44.992495 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:44.992496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:44.992620 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:44.993310 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:44.993319 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:44.995361 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:44.997211 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:44.997220 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:44.997223 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:44.997579 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:44.997792 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:44.997793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:44.997793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:44.997949 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:44.998206 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:44.998213 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:44.998396 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 20:56:44.998516 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 20:56:44.998706 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":89,"threads_started":1}
I20260808 20:56:44.999653 25630 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33823
I20260808 20:56:44.999857 25630 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:45.000144 25630 heartbeater.cc:511] Master 127.15.232.190:33823 requested a full tablet report, sending...
I20260808 20:56:45.000779 25336 ts_manager.cc:194] Registered new tserver with Master: 5288b5f446634ee39afe9dff54506397 (127.15.232.129:33485)
I20260808 20:56:45.001418 25336 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:49781
WARNING: no policy specified for oryx/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab.
Entry for principal oryx/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab.
I20260808 20:56:45.024366 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33823
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab
--principal=oryx/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:45.112711 25637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:45.113006 25637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:45.113034 25637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:45.114266 25637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:45.114313 25637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:45.114341 25637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:45.115998 25637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab
--principal=oryx/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:45.116271 25637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:45.116541 25637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:45.122319 25644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:45.122362 25637 server_base.cc:1034] running on GCE node
W20260808 20:56:45.122249 25647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:45.122255 25652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:45.122853 25637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:45.123159 25637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:45.124305 25637 hybrid_clock.cc:648] HybridClock initialized: now 1786222605124290 us; error 40 us; skew 500 ppm
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:45.126984 25637 init.cc:377] Logged in from keytab as oryx/127.15.232.130@KRBTEST.COM (short username oryx)
I20260808 20:56:45.127556 25637 webserver.cc:460] Webserver started at http://127.15.232.130:39199/ using document root <none> and password file <none>
I20260808 20:56:45.127782 25637 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:45.127822 25637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:45.127916 25637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:45.128952 25637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "99dd2c1983554bb78e7040f3dfbebf0b"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "5b76b98e1abd46f2f05170174389afef"
server_key_iv: "1bfb256148e2a93edd8629e61ed21d9d"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.129283 25637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "99dd2c1983554bb78e7040f3dfbebf0b"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "5b76b98e1abd46f2f05170174389afef"
server_key_iv: "1bfb256148e2a93edd8629e61ed21d9d"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.130723 25637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:45.131508 25665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.131717 25637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:45.131798 25637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "99dd2c1983554bb78e7040f3dfbebf0b"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "5b76b98e1abd46f2f05170174389afef"
server_key_iv: "1bfb256148e2a93edd8629e61ed21d9d"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.131896 25637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:45.143134 25637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:45.145366 25637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:45.145541 25637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:45.145798 25637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:45.146147 25637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:45.146204 25637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.146240 25637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:45.146268 25637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.153221 25637 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:32911
I20260808 20:56:45.153307 25832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:32911 every 8 connection(s)
I20260808 20:56:45.153687 25637 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:45.160614 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 25637
I20260808 20:56:45.160730 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:45.160961 16290 external_mini_cluster.cc:1428] Setting key 715c93a430976cd8da7b5a3d69a385c5
I20260808 20:56:45.164470 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.154513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:59593 (local address 127.15.232.190:33823)
0808 20:56:45.154562 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.154564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.154831 (+ 267us) server_negotiation.cc:407] Connection header received
0808 20:56:45.155491 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.155493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.155510 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.155592 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.156587 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.157138 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.158186 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.158380 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.158440 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.160999 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.161000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.161001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.161009 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.162761 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.163299 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.163300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.163300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.163357 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.163707 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.163708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.163709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.163883 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.163963 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.164242 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.164351 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 20:56:45.164507 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.154505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.130:59593)
0808 20:56:45.154660 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.154717 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.155261 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.155657 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.155660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.156269 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.156472 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.156477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.157231 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.157232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.158104 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.158107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.158461 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.158462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.158565 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.159157 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.159166 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.160869 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.162849 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.162857 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.162860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.163219 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.163426 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.163427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.163427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.163624 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.163959 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.163964 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.164103 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.164193 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.164323 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260808 20:56:45.165066 25833 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33823
I20260808 20:56:45.165203 25833 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:45.165413 25833 heartbeater.cc:511] Master 127.15.232.190:33823 requested a full tablet report, sending...
I20260808 20:56:45.165818 25336 ts_manager.cc:194] Registered new tserver with Master: 99dd2c1983554bb78e7040f3dfbebf0b (127.15.232.130:32911)
I20260808 20:56:45.166210 25336 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:59593
WARNING: no policy specified for oryx/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab.
Entry for principal oryx/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab.
I20260808 20:56:45.191015 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33823
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab
--principal=oryx/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:45.278841 25840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:45.279145 25840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:45.279192 25840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:45.280941 25840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:45.281040 25840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:45.281092 25840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:45.283108 25840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab
--principal=oryx/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:45.283473 25840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:45.283767 25840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:45.289623 25846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:45.289628 25849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:45.289784 25840 server_base.cc:1034] running on GCE node
W20260808 20:56:45.289628 25847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:45.290184 25840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:45.290468 25840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:45.291639 25840 hybrid_clock.cc:648] HybridClock initialized: now 1786222605291595 us; error 48 us; skew 500 ppm
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:45.294354 25840 init.cc:377] Logged in from keytab as oryx/127.15.232.131@KRBTEST.COM (short username oryx)
I20260808 20:56:45.294831 25840 webserver.cc:460] Webserver started at http://127.15.232.131:45097/ using document root <none> and password file <none>
I20260808 20:56:45.295090 25840 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:45.295142 25840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:45.295290 25840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:45.296545 25840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "edb4e94b0b374868a5763a1af93e77e2"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "f8c2e6a3f126ee4270a3b202aaeee27b"
server_key_iv: "b116efacaee8839eb056c6500a13eba0"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.296976 25840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "edb4e94b0b374868a5763a1af93e77e2"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "f8c2e6a3f126ee4270a3b202aaeee27b"
server_key_iv: "b116efacaee8839eb056c6500a13eba0"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.298696 25840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:45.299726 25858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.300454 25840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:45.300700 25840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "edb4e94b0b374868a5763a1af93e77e2"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "f8c2e6a3f126ee4270a3b202aaeee27b"
server_key_iv: "b116efacaee8839eb056c6500a13eba0"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.300804 25840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:45.325649 25840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:45.328227 25840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:45.328415 25840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:45.328679 25840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:45.329039 25840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:45.329114 25840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.329160 25840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:45.329190 25840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.335723 25840 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:35453
I20260808 20:56:45.335764 26028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:35453 every 8 connection(s)
I20260808 20:56:45.336134 25840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:45.337502 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 25840
I20260808 20:56:45.337634 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:45.337860 16290 external_mini_cluster.cc:1428] Setting key d2e8cc89db0cc4685a89982880c4c851
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:45.346055 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.336937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:45261 (local address 127.15.232.190:33823)
0808 20:56:45.337018 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.337020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.337173 (+ 153us) server_negotiation.cc:407] Connection header received
0808 20:56:45.337790 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.337792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.337809 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.337898 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.339180 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.339753 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.340820 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.341017 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.341074 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.343354 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.343355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.343355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.343363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.344700 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.345130 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.345131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.345131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.345192 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.345451 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.345452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.345452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.345576 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.345633 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.345894 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.345985 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 20:56:45.346215 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.336850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.131:45261)
0808 20:56:45.337040 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.337085 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.337605 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.338012 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.338017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.338877 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.339091 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.339097 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.339844 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.339845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.340725 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.340728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.341094 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.341095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.341199 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.341738 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.341747 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.343238 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.344784 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.344791 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.344793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.345037 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.345263 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.345264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.345264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.345379 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.345634 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.345639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.345769 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.345853 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.345985 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260808 20:56:45.346732 26030 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33823
I20260808 20:56:45.346839 26030 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:45.346998 26030 heartbeater.cc:511] Master 127.15.232.190:33823 requested a full tablet report, sending...
I20260808 20:56:45.347369 25336 ts_manager.cc:194] Registered new tserver with Master: edb4e94b0b374868a5763a1af93e77e2 (127.15.232.131:35453)
I20260808 20:56:45.347769 25336 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:45261
I20260808 20:56:45.349651 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.232.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.189.keytab.
Entry for principal oryx/127.15.232.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.189.keytab.
I20260808 20:56:45.378973 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.189:38629
--webserver_interface=127.15.232.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.189.keytab
--principal=oryx/127.15.232.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:33823,127.15.232.189:38629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:45.510170 26038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:45.510406 26038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:45.510443 26038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:45.511925 26038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:45.511976 26038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:45.511993 26038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:45.512007 26038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:45.512053 26038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:45.514201 26038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:33823,127.15.232.189:38629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.189.keytab
--principal=oryx/127.15.232.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.189:38629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.232.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
W20260808 20:56:45.514401 26038 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.232.190:33823,127.15.232.189:38629'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 20:56:45.514626 26038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:45.514838 26038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:45.520563 26056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:45.520563 26049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:45.520685 26048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:45.520912 26038 server_base.cc:1034] running on GCE node
I20260808 20:56:45.521210 26038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:45.521510 26038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:45.522687 26038 hybrid_clock.cc:648] HybridClock initialized: now 1786222605522669 us; error 48 us; skew 500 ppm
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:45.525305 26038 init.cc:377] Logged in from keytab as oryx/127.15.232.189@KRBTEST.COM (short username oryx)
I20260808 20:56:45.525815 26038 webserver.cc:460] Webserver started at http://127.15.232.189:39589/ using document root <none> and password file <none>
I20260808 20:56:45.526017 26038 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:45.526058 26038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:45.526150 26038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:45.527187 26038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data/instance:
uuid: "98491c8938bf48b6b78207f7f2ba7638"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "0c9d2a7fe4e305fb46b7c4670b93c679"
server_key_iv: "ed93b1d84e82804cf74447b89a0fab41"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.527580 26038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal/instance:
uuid: "98491c8938bf48b6b78207f7f2ba7638"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "0c9d2a7fe4e305fb46b7c4670b93c679"
server_key_iv: "ed93b1d84e82804cf74447b89a0fab41"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.528968 26038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:45.529729 26068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.529887 26038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:45.529983 26038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal
uuid: "98491c8938bf48b6b78207f7f2ba7638"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "0c9d2a7fe4e305fb46b7c4670b93c679"
server_key_iv: "ed93b1d84e82804cf74447b89a0fab41"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.530053 26038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:45.540266 26038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:45.542194 26038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:45.542311 26038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:45.546329 26038 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.189:38629
I20260808 20:56:45.546386 26140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.189:38629 every 8 connection(s)
I20260808 20:56:45.546733 26038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
I20260808 20:56:45.547890 26144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:45.549678 26144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.189@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:45.556313 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 26038
I20260808 20:56:45.559890 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.550448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.15.232.190:33823)
0808 20:56:45.550538 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.550540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.550802 (+ 262us) server_negotiation.cc:407] Connection header received
0808 20:56:45.551245 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.551246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.551257 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.551321 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.552068 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.552623 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.553620 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.553819 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.553878 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.556285 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.556286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.556287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.556295 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.558027 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.558711 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.558713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.558713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.558786 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.559074 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.559075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.559075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.559243 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.559331 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.559702 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.559793 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 20:56:45.560119 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.550423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:54184)
0808 20:56:45.550629 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.550672 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.551131 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.551401 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.551405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.551753 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.551954 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.551958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.552711 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.552712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.553533 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.553535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.553899 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.553900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.554007 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.554580 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.554606 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.556141 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.558281 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.558288 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.558291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.558615 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.558867 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.558867 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.558868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.558989 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.559320 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.559326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.559559 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.559679 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.559855 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":120,"threads_started":1}
I20260808 20:56:45.560858 26144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.189@KRBTEST.COM
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.189@KRBTEST.COM for oryx/127.15.232.189@KRBTEST.COM
I20260808 20:56:45.566495 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.556994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45400 (local address 127.15.232.189:38629)
0808 20:56:45.557211 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.557217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.557245 (+ 28us) server_negotiation.cc:407] Connection header received
0808 20:56:45.557269 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.557271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.557285 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.557380 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.558154 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.558966 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.559805 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.560259 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.560334 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.563019 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.563028 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.563028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.563035 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.564862 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.565399 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.565402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.565403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.565466 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.565690 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.565691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.565691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.565892 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.566033 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.566248 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.566377 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":127,"threads_started":1}
I20260808 20:56:45.568902 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.561148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.0.0.1:45416)
0808 20:56:45.561374 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.561386 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.561439 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.561572 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.561572 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.561729 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.561772 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.561773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.562681 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.562682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.563171 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.563172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.563648 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.563649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.563716 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.564145 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.564149 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.565390 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.567841 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.567841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.567842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.568008 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.568218 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.568219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.568220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.568275 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.568535 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.568536 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.568574 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.568666 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.568767 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260808 20:56:45.568979 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.561211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.15.232.189:38629)
0808 20:56:45.561387 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.561389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.561423 (+ 34us) server_negotiation.cc:407] Connection header received
0808 20:56:45.561494 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.561495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.561503 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.561569 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.561848 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.562614 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.563257 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.563560 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.563648 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.565507 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.565508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.565508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.565515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.567744 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.568081 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.568082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.568082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.568150 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.568348 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.568349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.568349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.568461 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.568533 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.568689 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.568821 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":158,"thread_start_us":128,"threads_started":1}
I20260808 20:56:45.570382 26144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Bootstrap starting.
I20260808 20:56:45.571221 26144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:45.571580 26144 log.cc:824] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:45.572372 26144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: No bootstrap required, opened a new log
I20260808 20:56:45.573859 26144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } }
I20260808 20:56:45.573995 26144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:45.574046 26144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98491c8938bf48b6b78207f7f2ba7638, State: Initialized, Role: FOLLOWER
I20260808 20:56:45.574185 26144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } }
I20260808 20:56:45.574472 26153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } } }
I20260808 20:56:45.574522 26144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:45.574569 26153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 20:56:45.576305 26170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 20:56:45.576391 26170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 20:56:45.578325 26038 master_runner.cc:428] Detected that this master 98491c8938bf48b6b78207f7f2ba7638 is joining an existing cluster
I20260808 20:56:45.578423 26038 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.232.189:38629
I20260808 20:56:45.587306 26178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.579564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.0.0.1:45422)
0808 20:56:45.579774 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.579805 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.579854 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.580022 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.580023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.580299 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.580384 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.580387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.581353 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.581353 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.582308 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.582311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.582720 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.582725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.582762 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.583314 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.583321 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.583900 (+ 579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.585977 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.585979 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.585979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.586314 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.586527 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.586528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.586528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.586581 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.586902 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.586902 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.586940 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.587011 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.587156 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":33,"thread_start_us":113,"threads_started":1}
I20260808 20:56:45.587327 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.579766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.15.232.189:38629)
0808 20:56:45.579831 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.579833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.579857 (+ 24us) server_negotiation.cc:407] Connection header received
0808 20:56:45.579918 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.579918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.579929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.579981 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.580476 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.581254 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.582408 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.582628 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.582660 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.583998 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.584000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.584000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.584007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.585872 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.586396 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.586397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.586398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.586456 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.586664 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.586664 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.586665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.586793 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.586879 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.587065 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.587225 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":45}
I20260808 20:56:45.587698 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.579686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.15.232.190:33823)
0808 20:56:45.579732 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.579735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.580060 (+ 325us) server_negotiation.cc:407] Connection header received
0808 20:56:45.580091 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.580092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.580104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.580180 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.580513 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.581261 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.582419 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.582863 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.583316 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.584447 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.584448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.584449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.584456 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.586091 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.586513 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.586514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.586514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.586587 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.586858 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.586860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.586860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.587007 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.587069 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.587261 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.587351 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 20:56:45.588089 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.579566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:54186)
0808 20:56:45.579956 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.579965 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.580016 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.580182 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.580183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.580365 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.580415 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.580417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.581353 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.581353 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.582308 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.582311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.583337 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.583338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.583412 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.583853 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.583858 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.584333 (+ 475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.586168 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.586169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.586170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.586429 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.586669 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.586670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.586671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.586775 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.587098 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.587099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.587138 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.587889 (+ 751us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.588001 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":41,"thread_start_us":108,"threads_started":1}
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222604, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.190@KRBTEST.COM for oryx/127.15.232.189@KRBTEST.COM
I20260808 20:56:45.598506 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.590401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.0.0.1:45424)
0808 20:56:45.590600 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.590651 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.590733 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.590870 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.590873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.591226 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.591293 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.591296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.591985 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.591986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.592961 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.592964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.593300 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.593301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.593388 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.593816 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.593822 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.595337 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.597063 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.597074 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.597076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.597395 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.597661 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.597662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.597663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.597772 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.598147 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.598156 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.598241 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.598318 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.598428 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
I20260808 20:56:45.598546 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.590485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.15.232.189:38629)
0808 20:56:45.590527 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.590529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.590724 (+ 195us) server_negotiation.cc:407] Connection header received
0808 20:56:45.590808 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.590809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.590818 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.590864 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.591368 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.591905 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.593053 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.593227 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.593277 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.595451 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.595453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.595453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.595461 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.596878 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.597498 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.597499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.597500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.597570 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.597866 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.597867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.597867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.598055 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.598128 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.598361 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.598449 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:56:45.599306 25336 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.232.189:38629: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638"
member_type: NON_VOTER
last_known_addr {
host: "127.15.232.189"
port: 38629
}
attrs {
promote: true
}
}
dest_uuid: "ea87f99a39bd455392ccdd6f0fe2909e"
cas_config_opid_index: -1
I20260808 20:56:45.599749 25336 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } }
I20260808 20:56:45.600276 26183 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 98491c8938bf48b6b78207f7f2ba7638 (127.15.232.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } }
I20260808 20:56:45.600713 26108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:45.601272 26183 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.232.189:38629
I20260808 20:56:45.601366 26183 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } }
I20260808 20:56:45.601436 26183 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:45.601630 26038 master.cc:537] Master@127.15.232.189:38629 shutting down...
I20260808 20:56:45.601722 26108 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Refusing update from remote peer ea87f99a39bd455392ccdd6f0fe2909e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 20:56:45.602154 26183 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 20:56:45.603129 25317 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 20:56:45.604521 25317 consensus_peers.cc:487] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e -> Peer 98491c8938bf48b6b78207f7f2ba7638 (127.15.232.189:38629): Couldn't send request to peer 98491c8938bf48b6b78207f7f2ba7638. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 20:56:45.605059 26038 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:45.605158 26038 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:45.605196 26038 tablet_replica.cc:330] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: stopping tablet replica
I20260808 20:56:45.619253 26038 master.cc:552] Master@127.15.232.189:38629 shutdown complete.
I20260808 20:56:45.619330 26038 master_runner.cc:305] Clearing existing system tablet
I20260808 20:56:45.619518 26038 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:45.619568 26038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:45.620604 26038 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:45.621397 26188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.621542 26038 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:45.621590 26038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.621618 26038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal
uuid: "98491c8938bf48b6b78207f7f2ba7638"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "0c9d2a7fe4e305fb46b7c4670b93c679"
server_key_iv: "ed93b1d84e82804cf74447b89a0fab41"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.622699 26038 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:45.624368 26038 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 20:56:45.624444 26038 log.cc:1196] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 20:56:45.624718 26038 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Deleting consensus metadata
I20260808 20:56:45.624861 26038 master_runner.cc:315] Copying system tablet from 127.15.232.190:33823
I20260808 20:56:45.625588 26038 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: tablet copy: Beginning tablet copy session from remote peer at address 127.15.232.190:33823
I20260808 20:56:45.632319 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.625964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:54188)
0808 20:56:45.626089 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.626101 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.626169 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.626329 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.626330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.626500 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.626560 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.626561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.627511 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.627512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.628070 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.628071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.628562 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.628563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.628617 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.629142 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.629146 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.629451 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.631221 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.631221 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.631222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.631411 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.631716 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.631717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.631717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.631778 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.632049 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.632049 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.632089 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.632146 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.632234 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":85,"threads_started":1}
I20260808 20:56:45.632366 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.626047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54188 (local address 127.15.232.190:33823)
0808 20:56:45.626100 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.626102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.626172 (+ 70us) server_negotiation.cc:407] Connection header received
0808 20:56:45.626246 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.626247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.626260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.626318 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.626632 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.627410 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.628156 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.628465 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.628540 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.629548 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.629549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.629550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.629557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.631113 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.631575 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.631576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.631577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.631643 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.631843 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.631844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.631845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.631984 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.632056 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.632186 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.632269 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 20:56:45.633071 25373 tablet_copy_service.cc:140] P ea87f99a39bd455392ccdd6f0fe2909e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 98491c8938bf48b6b78207f7f2ba7638 ({username='oryx', principal='oryx/127.15.232.189@KRBTEST.COM'} at 127.0.0.1:54188)
I20260808 20:56:45.633229 25373 tablet_copy_service.cc:161] P ea87f99a39bd455392ccdd6f0fe2909e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 98491c8938bf48b6b78207f7f2ba7638 at {username='oryx', principal='oryx/127.15.232.189@KRBTEST.COM'} at 127.0.0.1:54188: session id = 98491c8938bf48b6b78207f7f2ba7638-00000000000000000000000000000000
I20260808 20:56:45.634033 25373 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 20:56:45.634743 26038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 20:56:45.636034 26038 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: tablet copy: Starting download of 0 data blocks...
I20260808 20:56:45.636219 26038 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: tablet copy: Starting download of 1 WAL segments...
I20260808 20:56:45.637502 26038 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 20:56:45.638432 25373 tablet_copy_service.cc:342] P ea87f99a39bd455392ccdd6f0fe2909e: Request end of tablet copy session 98491c8938bf48b6b78207f7f2ba7638-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.232.189@KRBTEST.COM'} at 127.0.0.1:54188
I20260808 20:56:45.638504 25373 tablet_copy_service.cc:433] P ea87f99a39bd455392ccdd6f0fe2909e: ending tablet copy session 98491c8938bf48b6b78207f7f2ba7638-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 98491c8938bf48b6b78207f7f2ba7638
I20260808 20:56:45.641440 26038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:45.643313 26206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:45.643509 26201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:45.649009 26209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:45.649143 26038 server_base.cc:1034] running on GCE node
I20260808 20:56:45.649353 26038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:45.649562 26038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:45.650770 26038 hybrid_clock.cc:648] HybridClock initialized: now 1786222605650756 us; error 33 us; skew 500 ppm
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:45.652716 26038 init.cc:377] Logged in from keytab as oryx/127.15.232.189@KRBTEST.COM (short username oryx)
I20260808 20:56:45.653082 26038 webserver.cc:460] Webserver started at http://127.15.232.189:36735/ using document root <none> and password file <none>
I20260808 20:56:45.653223 26038 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:45.653280 26038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:45.653995 26038 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:45.654592 26219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.654775 26038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:45.654834 26038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal
uuid: "98491c8938bf48b6b78207f7f2ba7638"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "0c9d2a7fe4e305fb46b7c4670b93c679"
server_key_iv: "ed93b1d84e82804cf74447b89a0fab41"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.654897 26038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:45.674396 26038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:45.676348 26038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:45.681442 26038 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.189:38629
I20260808 20:56:45.681484 26308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.189:38629 every 8 connection(s)
I20260808 20:56:45.681845 26038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
I20260808 20:56:45.682771 26311 sys_catalog.cc:263] Verifying existing consensus state
I20260808 20:56:45.683238 26311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Bootstrap starting.
I20260808 20:56:45.685768 26311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:56:45.685995 26311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Bootstrap complete.
I20260808 20:56:45.686200 26311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } }
I20260808 20:56:45.686273 26311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 98491c8938bf48b6b78207f7f2ba7638, State: Initialized, Role: LEARNER
I20260808 20:56:45.686299 26311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } }
I20260808 20:56:45.686475 26311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:45.686482 26317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } }
I20260808 20:56:45.686611 26317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: This master's current role is: LEARNER
I20260808 20:56:45.687994 26334 catalog_manager.cc:1259] Loaded cluster ID: 52d888ded2cb4dba8eba7e35c91af6e6
I20260808 20:56:45.688071 26334 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: loading cluster ID for follower catalog manager: success
I20260808 20:56:45.689258 26334 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: acquiring CA information for follower catalog manager: success
I20260808 20:56:45.689545 26334 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 20:56:45.725771 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.719121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.15.232.189:38629)
0808 20:56:45.719290 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.719292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.719299 (+ 7us) server_negotiation.cc:407] Connection header received
0808 20:56:45.719324 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.719324 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.719334 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.719409 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.720038 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.720757 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.721522 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.721757 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.721841 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.723155 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.723157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.723158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.723166 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.724564 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.724925 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.724927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.724927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.724991 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.725211 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.725211 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.725212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.725331 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.725442 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.725540 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.725636 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for oryx/127.15.232.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.188.keytab.
Entry for principal oryx/127.15.232.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.188.keytab.
I20260808 20:56:45.758654 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.188:46165
--webserver_interface=127.15.232.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.188.keytab
--principal=oryx/127.15.232.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:33823,127.15.232.189:38629,127.15.232.188:46165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:45.846623 26338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:45.846858 26338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:45.846894 26338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:45.848181 26338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:45.848249 26338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:45.848275 26338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:45.848306 26338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:45.848337 26338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:45.850108 26338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:33823,127.15.232.189:38629,127.15.232.188:46165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.188.keytab
--principal=oryx/127.15.232.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.188:46165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.232.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:45.850394 26338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:45.850626 26338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:45.856082 26350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:45.856083 26344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:45.856091 26345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:45.856267 26338 server_base.cc:1034] running on GCE node
I20260808 20:56:45.856570 26338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:45.856880 26338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:45.858067 26338 hybrid_clock.cc:648] HybridClock initialized: now 1786222605858046 us; error 33 us; skew 500 ppm
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:45.860627 26338 init.cc:377] Logged in from keytab as oryx/127.15.232.188@KRBTEST.COM (short username oryx)
I20260808 20:56:45.861126 26338 webserver.cc:460] Webserver started at http://127.15.232.188:35511/ using document root <none> and password file <none>
I20260808 20:56:45.861361 26338 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:45.861430 26338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:45.861589 26338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:45.862815 26338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data/instance:
uuid: "75a504a288f84b39a25966125c39c5f6"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "27809556af09dd9d16f8d0a38daf8bf7"
server_key_iv: "14db60a46c939460989c11cfe0297439"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.863224 26338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal/instance:
uuid: "75a504a288f84b39a25966125c39c5f6"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "27809556af09dd9d16f8d0a38daf8bf7"
server_key_iv: "14db60a46c939460989c11cfe0297439"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.864848 26338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:45.865979 26358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:45.866293 26338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:45.866431 26338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal
uuid: "75a504a288f84b39a25966125c39c5f6"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "27809556af09dd9d16f8d0a38daf8bf7"
server_key_iv: "14db60a46c939460989c11cfe0297439"
server_key_version: "encryptionkey@0"
I20260808 20:56:45.866496 26338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:45.880311 26338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:45.882449 26338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:45.882575 26338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:45.888645 26338 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.188:46165
I20260808 20:56:45.888676 26431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.188:46165 every 8 connection(s)
I20260808 20:56:45.889066 26338 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
I20260808 20:56:45.890334 26437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:45.892071 26437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } has no permanent_uuid. Determining permanent_uuid...
I20260808 20:56:45.896265 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 26338
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.188@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.188@KRBTEST.COM
I20260808 20:56:45.903282 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.892904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.15.232.190:33823)
0808 20:56:45.892996 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.892998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.893139 (+ 141us) server_negotiation.cc:407] Connection header received
0808 20:56:45.893715 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.893717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.893731 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.893798 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.894541 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.895108 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.896503 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.896713 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.896778 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.899677 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.899679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.899679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.899689 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.901617 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.902038 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.902039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.902040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.902114 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.902487 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.902489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.902489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.902642 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.902720 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.903076 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.903189 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 20:56:45.903581 26440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.892756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:54194)
0808 20:56:45.892989 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.893048 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.893595 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.893873 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.893877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.894176 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.894388 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.894393 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.895216 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.895217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.896376 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.896381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.896891 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.896893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.897104 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.897744 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.897753 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.899410 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.901720 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.901726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.901728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.901957 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.902217 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.902218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.902219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.902375 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.902727 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.902733 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.902920 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.903044 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.903208 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260808 20:56:45.904326 26437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } has no permanent_uuid. Determining permanent_uuid...
I20260808 20:56:45.906805 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.896980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46126 (local address 127.15.232.188:46165)
0808 20:56:45.897175 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.897180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.897202 (+ 22us) server_negotiation.cc:407] Connection header received
0808 20:56:45.897228 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.897231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.897255 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.897350 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.898063 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.899000 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.900009 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.900420 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.900478 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.903485 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.903493 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.903494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.903502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.905478 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.905861 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.905863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.905864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.905917 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.906139 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.906140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.906140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.906277 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.906385 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.906547 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.906688 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":159,"thread_start_us":122,"threads_started":1}
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.188@KRBTEST.COM for oryx/127.15.232.189@KRBTEST.COM
I20260808 20:56:45.911633 26440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.904839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.0.0.1:45452)
0808 20:56:45.904901 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.904914 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.904962 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.905138 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.905139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.905312 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.905362 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.905363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.906454 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.906454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.907196 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.907198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.907667 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.907668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.907726 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.908134 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.908139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.909273 (+ 1134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.910715 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.910716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.910716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.910875 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.911066 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.911067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.911067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.911125 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.911344 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.911345 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.911379 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.911463 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.911558 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 20:56:45.911666 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.904939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45452 (local address 127.15.232.189:38629)
0808 20:56:45.904991 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.904993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.905004 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:45.905034 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.905035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.905049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.905134 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.905485 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.906377 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.907279 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.907598 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.907671 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.909350 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.909351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.909351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.909357 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.910640 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.910952 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.910953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.910954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.911007 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.911182 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.911183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.911183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.911288 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.911343 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.911520 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.911600 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 20:56:45.912061 26437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222605, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.188@KRBTEST.COM for oryx/127.15.232.188@KRBTEST.COM
I20260808 20:56:45.918188 26440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.912362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.0.0.1:46136)
0808 20:56:45.912411 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.912421 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.912457 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.912566 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.912567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.912702 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.912726 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.912727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.913354 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.913354 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.913817 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.913818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.914202 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.914203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.914251 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.914606 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.914610 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.915842 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.917259 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.917260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.917260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.917419 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.917624 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.917624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.917624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.917676 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.917963 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.917964 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.917998 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.918049 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.918128 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 20:56:45.918251 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.912449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46136 (local address 127.15.232.188:46165)
0808 20:56:45.912486 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.912488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.912496 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:56:45.912513 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.912513 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.912520 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.912566 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.912817 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.913271 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.913896 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.914120 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.914202 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.915922 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.915924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.915924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.915930 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.917182 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.917490 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.917490 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.917491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.917547 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.917766 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.917776 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.917776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.917896 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.917963 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.918080 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.918179 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":21}
I20260808 20:56:45.919865 26437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: Bootstrap starting.
I20260808 20:56:45.920751 26437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:45.921058 26437 log.cc:824] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:45.921959 26437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: No bootstrap required, opened a new log
I20260808 20:56:45.923676 26437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } }
I20260808 20:56:45.923831 26437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:45.923880 26437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a504a288f84b39a25966125c39c5f6, State: Initialized, Role: FOLLOWER
I20260808 20:56:45.924026 26437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } }
I20260808 20:56:45.924476 26437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:45.924440 26443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } } }
I20260808 20:56:45.924579 26443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 20:56:45.926630 26460 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 20:56:45.926717 26460 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 20:56:45.929286 26338 master_runner.cc:428] Detected that this master 75a504a288f84b39a25966125c39c5f6 is joining an existing cluster
I20260808 20:56:45.929347 26338 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.232.188:46165
I20260808 20:56:45.937930 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.930831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45460 (local address 127.15.232.189:38629)
0808 20:56:45.930874 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.930876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.930883 (+ 7us) server_negotiation.cc:407] Connection header received
0808 20:56:45.930912 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.930913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.930925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.930996 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.931473 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.932268 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.933556 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.933853 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.933897 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.935215 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.935217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.935217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.935225 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.936735 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.937158 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.937159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.937160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.937213 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.937415 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.937416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.937416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.937528 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.937582 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.937786 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.937852 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 20:56:45.937988 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.930458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.0.0.1:45460)
0808 20:56:45.930707 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.930720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.930788 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.931018 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.931019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.931244 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.931329 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.931331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.932375 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.932376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.933447 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.933450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.933993 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.933994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.934080 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.934565 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.934570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.935051 (+ 481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.936825 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.936826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.936826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.937069 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.937260 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.937261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.937261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.937345 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.937599 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.937601 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.937678 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.937770 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.937881 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260808 20:56:45.952191 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.930727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.15.232.188:46165)
0808 20:56:45.930766 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.930768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.931184 (+ 416us) server_negotiation.cc:407] Connection header received
0808 20:56:45.931274 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.931275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.931284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.931311 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.931885 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.932616 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.933664 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.933921 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.933956 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.949289 (+ 15333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.949290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.949290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.949297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.951002 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.951360 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.951361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.951361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.951419 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.951652 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.951653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.951653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.951783 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.951863 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.952005 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.952099 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 20:56:45.952651 26476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.930692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.0.0.1:46152)
0808 20:56:45.931122 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.931132 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.931207 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.931368 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.931369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.931695 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.931798 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.931801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.932708 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.932708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.933576 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.933578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.934032 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.934033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.934091 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.948368 (+ 14277us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.948373 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.949153 (+ 780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.951100 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.951101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.951102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.951283 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.951516 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.951517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.951517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.951579 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.951853 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.951854 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.951895 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.952324 (+ 429us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.952437 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":17,"thread_start_us":61,"threads_started":1}
I20260808 20:56:45.953006 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.931002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:54204)
0808 20:56:45.931134 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.931142 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.931220 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.931406 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.931407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.931698 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.931802 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.931803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.932722 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.932723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.933710 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.933712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.934197 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.934198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.934259 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.948287 (+ 14028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.948326 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.949153 (+ 827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.951593 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.951595 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.951595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.951837 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.952101 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.952102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.952102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.952174 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.952548 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.952549 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.952643 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.952725 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.952876 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":141,"thread_start_us":75,"threads_started":1}
I20260808 20:56:45.953758 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.930472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54204 (local address 127.15.232.190:33823)
0808 20:56:45.930517 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.930520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.931193 (+ 673us) server_negotiation.cc:407] Connection header received
0808 20:56:45.931301 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.931302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.931313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.931344 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.931896 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.932636 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.933790 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.934085 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.934125 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.949307 (+ 15182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.949309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.949310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.949321 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.951482 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.951944 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.951953 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.951953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.952021 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.952252 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.952253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.952253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.952399 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.952470 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.953569 (+ 1099us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.953664 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 20:56:45.958720 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 25431
Aug 08 20:56:45 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222604, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.190@KRBTEST.COM for oryx/127.15.232.188@KRBTEST.COM
I20260808 20:56:45.971277 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.956267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.15.232.188:46165)
0808 20:56:45.956314 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.956316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.956327 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:45.956366 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.956367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.956377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.956425 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:45.956722 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.957185 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.957970 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.958167 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.958243 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.967785 (+ 9542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:45.967788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:45.967788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:45.967799 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:45.969761 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.970227 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.970229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.970230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.970303 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:45.970621 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:45.970622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:45.970622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:45.970790 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:45.970869 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.971072 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.971175 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:56:45.971323 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.956195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.0.0.1:46164)
0808 20:56:45.956252 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.956263 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.956305 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.956427 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.956427 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.956598 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:45.956642 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.956644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.957272 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.957273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.957883 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.957886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.958241 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.958242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.958321 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.965806 (+ 7485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:45.965815 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:45.967647 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:45.969878 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.969879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.969880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.970140 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.970439 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:45.970440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:45.970440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:45.970535 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:45.970872 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:45.970873 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:45.970936 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.971019 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.971163 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 20:56:45.972191 25336 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.232.188:46165: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "75a504a288f84b39a25966125c39c5f6"
member_type: NON_VOTER
last_known_addr {
host: "127.15.232.188"
port: 46165
}
attrs {
promote: true
}
}
dest_uuid: "ea87f99a39bd455392ccdd6f0fe2909e"
cas_config_opid_index: 5
I20260808 20:56:45.972491 25336 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } }
I20260808 20:56:45.972936 26183 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 75a504a288f84b39a25966125c39c5f6 (127.15.232.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.973544 26396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:45.973815 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:33485
--local_ip_for_outbound_sockets=127.15.232.129
--tserver_master_addrs=127.15.232.190:33823
--webserver_port=35599
--webserver_interface=127.15.232.129
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab
--principal=oryx/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.232.190:33823,127.15.232.189:38629
--tserver_master_addrs=127.15.232.190:33823,127.15.232.189:38629,127.15.232.188:46165 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:56:45.973945 26183 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.232.188:46165
I20260808 20:56:45.974030 26183 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.974114 26183 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:45.974766 26396 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 FOLLOWER]: Refusing update from remote peer ea87f99a39bd455392ccdd6f0fe2909e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 20:56:45.975205 26182 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:45.975795 26338 master.cc:537] Master@127.15.232.188:46165 shutting down...
I20260808 20:56:45.976815 26396 raft_consensus.cc:858] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } }
W20260808 20:56:45.976912 26396 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } }
I20260808 20:56:45.977422 26396 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 98491c8938bf48b6b78207f7f2ba7638 (127.15.232.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } }
I20260808 20:56:45.978648 26443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea87f99a39bd455392ccdd6f0fe2909e. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } }
I20260808 20:56:45.978734 26443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 20:56:45.978761 26443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } }
I20260808 20:56:45.978791 26443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 20:56:45.978814 26443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } }
I20260808 20:56:45.978845 26443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 20:56:45.980983 26182 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.981098 26182 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
W20260808 20:56:45.981693 25317 consensus_peers.cc:487] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e -> Peer 75a504a288f84b39a25966125c39c5f6 (127.15.232.188:46165): Couldn't send request to peer 75a504a288f84b39a25966125c39c5f6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 20:56:45.981986 26478 mvcc.cc:205] Tried to move back new op lower bound from 7316367789352787968 to 7316367789178949632. Current Snapshot: MvccSnapshot[applied={T|T < 7316367789189816320}]
I20260808 20:56:45.982589 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.973118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.15.232.189:38629)
0808 20:56:45.973202 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:45.973204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:45.976316 (+ 3112us) server_negotiation.cc:407] Connection header received
0808 20:56:45.976431 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:45.976432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:45.976458 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:45.976499 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:45.976743 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.977489 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.978982 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.979624 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.979682 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.979768 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:45.982444 (+ 2676us) server_negotiation.cc:299] Negotiation successful
0808 20:56:45.982509 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 20:56:45.982939 26338 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260808 20:56:45.983017 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.973011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.0.0.1:45468)
0808 20:56:45.976242 (+ 3231us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.976256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:45.976348 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:45.976598 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:45.976600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:45.976602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:45.976656 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.976658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.977608 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.977609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.978869 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:45.978873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:45.982192 (+ 3319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:45.982193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:45.982304 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:45.982343 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 20:56:45.982863 (+ 520us) client_negotiation.cc:240] Negotiation successful
0808 20:56:45.982931 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3209}
I20260808 20:56:45.983035 26338 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260808 20:56:45.983073 26338 tablet_replica.cc:330] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: stopping tablet replica
I20260808 20:56:45.983424 26261 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 LEARNER]: Refusing update from remote peer ea87f99a39bd455392ccdd6f0fe2909e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 20:56:45.983743 26489 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 20:56:45.984630 26261 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 75a504a288f84b39a25966125c39c5f6 (127.15.232.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.986496 26317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea87f99a39bd455392ccdd6f0fe2909e. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.986619 26317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: This master's current role is: LEARNER
I20260808 20:56:45.986654 26489 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: attempting to promote NON_VOTER 98491c8938bf48b6b78207f7f2ba7638 to VOTER
I20260808 20:56:45.986680 26317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: NON_VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: true } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.986734 26317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: This master's current role is: LEARNER
I20260808 20:56:45.986893 26489 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } }
I20260808 20:56:45.988088 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.987929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.0.0.1:46166)
0808 20:56:45.987981 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.988045 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 20:56:45.988493 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:45.988368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.0.0.1:46182)
0808 20:56:45.988433 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:45.988464 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 20:56:45.988557 26261 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 LEARNER]: Refusing update from remote peer ea87f99a39bd455392ccdd6f0fe2909e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260808 20:56:45.988566 25317 consensus_peers.cc:487] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e -> Peer 75a504a288f84b39a25966125c39c5f6 (127.15.232.188:46165): Couldn't send request to peer 75a504a288f84b39a25966125c39c5f6. Status: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46165: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 20:56:45.988746 26489 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 20:56:45.989444 26183 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 98491c8938bf48b6b78207f7f2ba7638 (127.15.232.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.990363 26183 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.990470 26183 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:45.990603 26489 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.990665 26489 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:45.990885 26261 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 98491c8938bf48b6b78207f7f2ba7638 (127.15.232.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.991741 26495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:45.991851 26495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:56:45.999673 26338 master.cc:552] Master@127.15.232.188:46165 shutdown complete.
I20260808 20:56:45.999751 26338 master_runner.cc:305] Clearing existing system tablet
I20260808 20:56:45.999933 26338 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:46.000026 26338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:46.000710 26338 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:46.001232 26498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.001441 26338 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:46.001497 26338 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:46.001546 26338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal
uuid: "75a504a288f84b39a25966125c39c5f6"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "27809556af09dd9d16f8d0a38daf8bf7"
server_key_iv: "14db60a46c939460989c11cfe0297439"
server_key_version: "encryptionkey@0"
I20260808 20:56:46.002183 26338 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:46.003556 26338 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 20:56:46.003652 26338 log.cc:1196] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 20:56:46.003931 26338 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: Deleting consensus metadata
I20260808 20:56:46.004066 26338 master_runner.cc:315] Copying system tablet from 127.15.232.190:33823
I20260808 20:56:46.004734 26338 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: tablet copy: Beginning tablet copy session from remote peer at address 127.15.232.190:33823
I20260808 20:56:46.011518 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.005147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.15.232.190:33823)
0808 20:56:46.005204 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.005206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.005346 (+ 140us) server_negotiation.cc:407] Connection header received
0808 20:56:46.005425 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.005426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.005437 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.005514 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.005860 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.006380 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.007116 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.007323 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.007403 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.008456 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.008458 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.008458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.008466 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.010231 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.010615 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.010616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.010617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.010682 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.010904 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.010905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.010906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.011036 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.011115 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.011282 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.011369 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 20:56:46.011870 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.005106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:54210)
0808 20:56:46.005254 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.005265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.005363 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.005520 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.005521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.005722 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.005769 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.005771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.006474 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.006475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.007022 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.007023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.007402 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.007403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.007508 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.008014 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.008020 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.008353 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.010328 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.010329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.010329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.010533 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.010756 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.010757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.010758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.010830 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.011106 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.011107 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.011158 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.011251 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.011369 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260808 20:56:46.012404 25373 tablet_copy_service.cc:140] P ea87f99a39bd455392ccdd6f0fe2909e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 75a504a288f84b39a25966125c39c5f6 ({username='oryx', principal='oryx/127.15.232.188@KRBTEST.COM'} at 127.0.0.1:54210)
I20260808 20:56:46.012477 25373 tablet_copy_service.cc:161] P ea87f99a39bd455392ccdd6f0fe2909e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 75a504a288f84b39a25966125c39c5f6 at {username='oryx', principal='oryx/127.15.232.188@KRBTEST.COM'} at 127.0.0.1:54210: session id = 75a504a288f84b39a25966125c39c5f6-00000000000000000000000000000000
I20260808 20:56:46.012969 25373 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 20:56:46.013546 26338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 20:56:46.014559 26338 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: tablet copy: Starting download of 0 data blocks...
I20260808 20:56:46.014750 26338 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: tablet copy: Starting download of 1 WAL segments...
I20260808 20:56:46.015558 26338 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 20:56:46.016433 25373 tablet_copy_service.cc:342] P ea87f99a39bd455392ccdd6f0fe2909e: Request end of tablet copy session 75a504a288f84b39a25966125c39c5f6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.232.188@KRBTEST.COM'} at 127.0.0.1:54210
I20260808 20:56:46.016521 25373 tablet_copy_service.cc:433] P ea87f99a39bd455392ccdd6f0fe2909e: ending tablet copy session 75a504a288f84b39a25966125c39c5f6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 75a504a288f84b39a25966125c39c5f6
I20260808 20:56:46.019140 26338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:46.020998 26507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:46.021030 26508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:46.023763 26338 server_base.cc:1034] running on GCE node
W20260808 20:56:46.024703 26511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:46.024914 26338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:46.025141 26338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:46.026278 26338 hybrid_clock.cc:648] HybridClock initialized: now 1786222606026267 us; error 31 us; skew 500 ppm
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:46.032234 26338 init.cc:377] Logged in from keytab as oryx/127.15.232.188@KRBTEST.COM (short username oryx)
I20260808 20:56:46.032603 26338 webserver.cc:460] Webserver started at http://127.15.232.188:37837/ using document root <none> and password file <none>
I20260808 20:56:46.032747 26338 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:46.032790 26338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:46.033492 26338 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 20:56:46.034098 26525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.034255 26338 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.034312 26338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal
uuid: "75a504a288f84b39a25966125c39c5f6"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "27809556af09dd9d16f8d0a38daf8bf7"
server_key_iv: "14db60a46c939460989c11cfe0297439"
server_key_version: "encryptionkey@0"
I20260808 20:56:46.034379 26338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:46.041049 26338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:46.042896 26338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:46.046166 26338 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.188:46165
I20260808 20:56:46.046491 26338 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
I20260808 20:56:46.046777 26596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.188:46165 every 8 connection(s)
I20260808 20:56:46.047531 26597 sys_catalog.cc:263] Verifying existing consensus state
I20260808 20:56:46.047936 26597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: Bootstrap starting.
I20260808 20:56:46.050220 26597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:56:46.050514 26597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: Bootstrap complete.
I20260808 20:56:46.050724 26597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } }
I20260808 20:56:46.050822 26597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 75a504a288f84b39a25966125c39c5f6, State: Initialized, Role: LEARNER
I20260808 20:56:46.050868 26597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } }
I20260808 20:56:46.051028 26597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:46.053092 26620 catalog_manager.cc:1259] Loaded cluster ID: 52d888ded2cb4dba8eba7e35c91af6e6
I20260808 20:56:46.053189 26620 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: loading cluster ID for follower catalog manager: success
I20260808 20:56:46.054023 26620 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: acquiring CA information for follower catalog manager: success
I20260808 20:56:46.054399 26620 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 20:56:46.055533 26604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:46.055904 26604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: This master's current role is: LEARNER
W20260808 20:56:46.108592 26477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:46.108870 26477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:46.108912 26477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:46.110231 26477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:46.110311 26477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:46.110360 26477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:46.112171 26477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab
--principal=oryx/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:33485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=35599
--tserver_master_addrs=127.15.232.190:33823,127.15.232.189:38629,127.15.232.188:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:46.112466 26477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:46.112839 26477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:46.119786 26635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:46.119796 26637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:46.120052 26633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:46.120265 26477 server_base.cc:1034] running on GCE node
I20260808 20:56:46.120497 26477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:46.120774 26477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:46.121958 26477 hybrid_clock.cc:648] HybridClock initialized: now 1786222606121918 us; error 55 us; skew 500 ppm
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:46.124699 26477 init.cc:377] Logged in from keytab as oryx/127.15.232.129@KRBTEST.COM (short username oryx)
I20260808 20:56:46.125259 26477 webserver.cc:460] Webserver started at http://127.15.232.129:35599/ using document root <none> and password file <none>
I20260808 20:56:46.125502 26477 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:46.125592 26477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:46.127135 26477 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:46.127990 26650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.128185 26477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:46.128311 26477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "5288b5f446634ee39afe9dff54506397"
format_stamp: "Formatted at 2026-08-08 20:56:44 on dist-test-slave-c12x"
server_key: "4638de6eac46eeb313c7cb7fdcca1c2f"
server_key_iv: "6d93f2c35bd6210a88511f6f312dea0e"
server_key_version: "encryptionkey@0"
I20260808 20:56:46.128721 26477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:46.142535 26477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:46.145557 26477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:46.146044 26477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:46.146435 26477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:46.146817 26477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:46.146878 26477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.146924 26477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:46.146955 26477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.157938 26477 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:33485
I20260808 20:56:46.157985 26811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:33485 every 8 connection(s)
I20260808 20:56:46.158412 26477 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:46.162431 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 26477
I20260808 20:56:46.162632 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 25637
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.188@KRBTEST.COM
I20260808 20:56:46.172528 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.159392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:56233 (local address 127.15.232.188:46165)
0808 20:56:46.159776 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.159778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.159788 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:46.160415 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.160416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.160432 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.160480 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.161576 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.162596 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.165846 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.166195 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.166234 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.168937 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.168940 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.168940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.168951 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.170823 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.171390 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.171391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.171392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.171475 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.171739 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.171740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.171740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.171886 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.171935 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.172285 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.172385 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":86,"threads_started":1}
I20260808 20:56:46.172622 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.159244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:55903 (local address 127.15.232.190:33823)
0808 20:56:46.159304 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.159306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.159906 (+ 600us) server_negotiation.cc:407] Connection header received
0808 20:56:46.160418 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.160420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.160436 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.160503 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.161568 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.162357 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.163861 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.164129 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.164167 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.167974 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.167977 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.167977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.167989 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.170726 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.171497 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.171499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.171500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.171578 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.171846 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.171847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.171847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.172025 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.172094 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.172368 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.172478 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 20:56:46.173406 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.159230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.15.232.129:56233)
0808 20:56:46.159485 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.159538 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.160280 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.160643 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.160648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.161215 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.161477 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.161479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.162688 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.162689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.165728 (+ 3039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.165731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.166309 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.166310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.166375 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.166877 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.166882 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.168805 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.170909 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.170911 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.170911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.171309 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.171536 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.171536 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.171537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.171678 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.171961 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.171967 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.172146 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.173080 (+ 934us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.173192 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":2254,"thread_start_us":97,"threads_started":1}
I20260808 20:56:46.173410 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.159230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.129:55903)
0808 20:56:46.159833 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.159843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.160280 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.160661 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.160662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.161218 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.161443 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.161446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.162500 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.162501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.163762 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.163765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.164248 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.164249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.164380 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.165314 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.165324 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.167803 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.170846 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.170860 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.170863 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.171349 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.171670 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.171671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.171672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.171751 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.172153 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.172154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.172213 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.172751 (+ 538us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.173025 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":330,"thread_start_us":111,"threads_started":1}
I20260808 20:56:46.174981 26812 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33823
I20260808 20:56:46.175138 26812 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:46.175545 26812 heartbeater.cc:511] Master 127.15.232.190:33823 requested a full tablet report, sending...
I20260808 20:56:46.176175 25336 ts_manager.cc:194] Re-registered known tserver with Master: 5288b5f446634ee39afe9dff54506397 (127.15.232.129:33485)
I20260808 20:56:46.176695 25336 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:55903
I20260808 20:56:46.182399 26813 heartbeater.cc:347] Connected to a master server at 127.15.232.188:46165
I20260808 20:56:46.182492 26813 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:46.182615 26813 heartbeater.cc:511] Master 127.15.232.188:46165 requested a full tablet report, sending...
I20260808 20:56:46.183090 26548 ts_manager.cc:194] Registered new tserver with Master: 5288b5f446634ee39afe9dff54506397 (127.15.232.129:33485)
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.189@KRBTEST.COM
I20260808 20:56:46.187971 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.159612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.15.232.129:45395)
0808 20:56:46.159950 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.159958 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.160307 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.160650 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.160651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.161208 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.161436 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.161442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.180525 (+ 19083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.180526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.181231 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.181235 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.181804 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.181806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.181912 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.182494 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.182508 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.184145 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.186294 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.186295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.186296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.186610 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.186913 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.186915 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.186915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.186985 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.187341 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.187342 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.187392 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.187592 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.187805 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":78,"threads_started":1}
I20260808 20:56:46.188067 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.159805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:45395 (local address 127.15.232.189:38629)
0808 20:56:46.159879 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.159881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.160020 (+ 139us) server_negotiation.cc:407] Connection header received
0808 20:56:46.160530 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.160531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.160546 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.160615 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.161624 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.162168 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.181359 (+ 19191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.181702 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.181779 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.184249 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.184251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.184251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.184262 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.186198 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.186727 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.186729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.186729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.186801 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.187069 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.187070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.187071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.187270 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.187351 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.187647 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.187748 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 20:56:46.188237 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:32911
--local_ip_for_outbound_sockets=127.15.232.130
--tserver_master_addrs=127.15.232.190:33823
--webserver_port=39199
--webserver_interface=127.15.232.130
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab
--principal=oryx/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.232.190:33823,127.15.232.189:38629
--tserver_master_addrs=127.15.232.190:33823,127.15.232.189:38629,127.15.232.188:46165 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:56:46.188542 26814 heartbeater.cc:347] Connected to a master server at 127.15.232.189:38629
I20260808 20:56:46.188629 26814 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:46.188696 26814 heartbeater.cc:511] Master 127.15.232.189:38629 requested a full tablet report, sending...
I20260808 20:56:46.189129 26242 ts_manager.cc:194] Registered new tserver with Master: 5288b5f446634ee39afe9dff54506397 (127.15.232.129:33485)
W20260808 20:56:46.279428 26822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:46.279807 26822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:46.279850 26822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:46.281275 26822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:46.281337 26822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:46.281392 26822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:46.283068 26822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab
--principal=oryx/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:32911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=39199
--tserver_master_addrs=127.15.232.190:33823,127.15.232.189:38629,127.15.232.188:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:46.283339 26822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:46.283612 26822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 20:56:46.289335 26822 server_base.cc:1034] running on GCE node
W20260808 20:56:46.289357 26831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:46.289333 26833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:46.289469 26836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:46.289685 26822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:46.289920 26822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:46.291077 26822 hybrid_clock.cc:648] HybridClock initialized: now 1786222606291040 us; error 56 us; skew 500 ppm
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:46.293699 26822 init.cc:377] Logged in from keytab as oryx/127.15.232.130@KRBTEST.COM (short username oryx)
I20260808 20:56:46.294211 26822 webserver.cc:460] Webserver started at http://127.15.232.130:39199/ using document root <none> and password file <none>
I20260808 20:56:46.294441 26822 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:46.294519 26822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:46.296126 26822 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:46.296748 26847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.296952 26822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:46.297029 26822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "99dd2c1983554bb78e7040f3dfbebf0b"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "5b76b98e1abd46f2f05170174389afef"
server_key_iv: "1bfb256148e2a93edd8629e61ed21d9d"
server_key_version: "encryptionkey@0"
I20260808 20:56:46.297371 26822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:46.306751 26822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:46.308799 26822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:46.308923 26822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:46.309151 26822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:46.309427 26822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:46.309456 26822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.309479 26822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:46.309491 26822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.315834 26822 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:32911
I20260808 20:56:46.315901 27017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:32911 every 8 connection(s)
I20260808 20:56:46.316282 26822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.189@KRBTEST.COM
I20260808 20:56:46.324747 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 26822
I20260808 20:56:46.324921 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 25840
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.188@KRBTEST.COM
I20260808 20:56:46.330935 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.317622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:49137 (local address 127.15.232.188:46165)
0808 20:56:46.317692 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.317694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.317740 (+ 46us) server_negotiation.cc:407] Connection header received
0808 20:56:46.318716 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.318717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.318728 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.318789 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.319689 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.320450 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.322493 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.322802 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.322879 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.326924 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.326927 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.326927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.326938 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.329091 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.329675 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.329677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.329677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.329745 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.330056 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.330057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.330057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.330219 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.330267 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.330719 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.330828 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 20:56:46.331038 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.317140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:39503 (local address 127.15.232.190:33823)
0808 20:56:46.317237 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.317239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.317493 (+ 254us) server_negotiation.cc:407] Connection header received
0808 20:56:46.318843 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.318844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.318854 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.318921 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.319663 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.320183 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.321779 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.322098 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.322174 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.327489 (+ 5315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.327491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.327491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.327501 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.329830 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.330223 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.330225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.330225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.330288 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.330485 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.330486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.330487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.330618 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.330664 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.330863 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.330941 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 20:56:46.331636 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.317142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.130:39503)
0808 20:56:46.317346 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.317397 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.318524 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.318928 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.318929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.319292 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.319565 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.319570 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.320294 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.320295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.321680 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.321683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.322191 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.322192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.322332 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.323002 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.323011 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.326792 (+ 3781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.329910 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.329911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.329911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.330135 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.330361 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.330362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.330362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.330418 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.330713 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.330713 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.330751 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.331132 (+ 381us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.331351 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":17,"spinlock_wait_cycles":7424,"thread_start_us":99,"threads_started":1}
I20260808 20:56:46.331768 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.317440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.15.232.130:49137)
0808 20:56:46.317681 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.317693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.318541 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.318857 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.318861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.319321 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.319566 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.319572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.321785 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.321786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.322404 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.322406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.322889 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.322889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.322993 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.323304 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.323309 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.326385 (+ 3076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.329207 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.329217 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.329220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.329555 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.329821 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.329822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.329822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.329973 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.330318 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.330326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.330519 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.331039 (+ 520us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.331402 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":154,"thread_start_us":71,"threads_started":1}
I20260808 20:56:46.332289 27022 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33823
I20260808 20:56:46.332393 27022 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:46.332624 27022 heartbeater.cc:511] Master 127.15.232.190:33823 requested a full tablet report, sending...
I20260808 20:56:46.332890 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.317140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:37053 (local address 127.15.232.189:38629)
0808 20:56:46.317195 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.317197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.317493 (+ 296us) server_negotiation.cc:407] Connection header received
0808 20:56:46.318651 (+ 1158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.318653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.318667 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.318725 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.319688 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.320445 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.321778 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.322089 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.322166 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.324941 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.324943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.324943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.324952 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.330052 (+ 5100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.330554 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.330556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.330557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.330631 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.330902 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.330903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.330904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.331093 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.331146 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.332611 (+ 1465us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.332728 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 20:56:46.333011 25336 ts_manager.cc:194] Re-registered known tserver with Master: 99dd2c1983554bb78e7040f3dfbebf0b (127.15.232.130:32911)
I20260808 20:56:46.333436 25336 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:39503
I20260808 20:56:46.333544 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.317142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.15.232.130:37053)
0808 20:56:46.317346 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.317397 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.318524 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.318857 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.318861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.319333 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.319566 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.319572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.320527 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.320528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.321680 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.321683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.322524 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.322524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.322597 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.323152 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.323157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.324845 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.330170 (+ 5325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.330171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.330172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.330448 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.330725 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.330726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.330726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.330808 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.332276 (+ 1468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.332277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.332342 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.333222 (+ 880us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.333371 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":14,"thread_start_us":95,"threads_started":1}
I20260808 20:56:46.333987 27026 heartbeater.cc:347] Connected to a master server at 127.15.232.189:38629
I20260808 20:56:46.334064 27026 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:46.334255 27026 heartbeater.cc:511] Master 127.15.232.189:38629 requested a full tablet report, sending...
I20260808 20:56:46.334621 26242 ts_manager.cc:194] Registered new tserver with Master: 99dd2c1983554bb78e7040f3dfbebf0b (127.15.232.130:32911)
I20260808 20:56:46.341331 27024 heartbeater.cc:347] Connected to a master server at 127.15.232.188:46165
I20260808 20:56:46.341432 27024 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:46.341542 27024 heartbeater.cc:511] Master 127.15.232.188:46165 requested a full tablet report, sending...
I20260808 20:56:46.341951 26548 ts_manager.cc:194] Registered new tserver with Master: 99dd2c1983554bb78e7040f3dfbebf0b (127.15.232.130:32911)
I20260808 20:56:46.345400 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:35453
--local_ip_for_outbound_sockets=127.15.232.131
--tserver_master_addrs=127.15.232.190:33823
--webserver_port=45097
--webserver_interface=127.15.232.131
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab
--principal=oryx/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.232.190:33823,127.15.232.189:38629
--tserver_master_addrs=127.15.232.190:33823,127.15.232.189:38629,127.15.232.188:46165 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:46.451469 27037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:46.451720 27037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:46.451750 27037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:46.453045 27037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:46.453089 27037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:46.453118 27037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:46.454671 27037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab
--principal=oryx/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:35453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=45097
--tserver_master_addrs=127.15.232.190:33823,127.15.232.189:38629,127.15.232.188:46165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:46.454944 27037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:46.455194 27037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:46.460611 27046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:46.460608 27043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:46.461167 27044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:46.461184 27037 server_base.cc:1034] running on GCE node
I20260808 20:56:46.461529 27037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:46.461782 27037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:46.462875 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.459363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.0.0.1:56602)
0808 20:56:46.459421 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.459471 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.459525 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.459785 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.459786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.459789 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:46.459841 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.459844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.461044 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.461045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.462150 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.462153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.462696 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.462697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.462741 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.462745 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.462786 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.462821 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 20:56:46.462889 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.459426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.15.232.188:46165)
0808 20:56:46.459587 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.459589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.459601 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:56:46.459635 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.459636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.459671 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.459757 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:46.460024 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.460855 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.462231 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.462622 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.462686 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.462731 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.462805 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.462838 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260808 20:56:46.463078 27037 hybrid_clock.cc:648] HybridClock initialized: now 1786222606463063 us; error 29 us; skew 500 ppm
I20260808 20:56:46.463212 26562 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 20:56:46.463375 26604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea87f99a39bd455392ccdd6f0fe2909e. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:46.463532 26604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: This master's current role is: LEARNER
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:46.465813 27037 init.cc:377] Logged in from keytab as oryx/127.15.232.131@KRBTEST.COM (short username oryx)
I20260808 20:56:46.466300 27037 webserver.cc:460] Webserver started at http://127.15.232.131:45097/ using document root <none> and password file <none>
I20260808 20:56:46.466548 27037 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:46.466641 27037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:46.468137 27037 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:46.468763 27058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.468978 27037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:46.469090 27037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "edb4e94b0b374868a5763a1af93e77e2"
format_stamp: "Formatted at 2026-08-08 20:56:45 on dist-test-slave-c12x"
server_key: "f8c2e6a3f126ee4270a3b202aaeee27b"
server_key_iv: "b116efacaee8839eb056c6500a13eba0"
server_key_version: "encryptionkey@0"
I20260808 20:56:46.469470 27037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:46.498394 27037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:46.501019 27037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:46.501221 27037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:46.501497 27037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:46.501844 27037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:46.501909 27037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.501955 27037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:46.501979 27037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:46.508994 27037 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:35453
I20260808 20:56:46.509044 27224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:35453 every 8 connection(s)
I20260808 20:56:46.509516 27037 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:46.511987 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 27037
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.189@KRBTEST.COM
I20260808 20:56:46.522846 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.513339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.15.232.188:46165)
0808 20:56:46.513563 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.513565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.514330 (+ 765us) server_negotiation.cc:407] Connection header received
0808 20:56:46.514428 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.514429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.514442 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.514481 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.515428 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.516179 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.517272 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.517574 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.517636 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.519140 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.519142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.519142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.519150 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.521300 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.521758 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.521759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.521760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.521848 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.522145 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.522146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.522146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.522293 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.522372 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.522630 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.522733 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":151,"threads_started":1}
I20260808 20:56:46.523253 27240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.513869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43678 (local address 127.15.232.190:33823)
0808 20:56:46.514057 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.514059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.514325 (+ 266us) server_negotiation.cc:407] Connection header received
0808 20:56:46.514414 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.514414 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.514428 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.514477 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.515416 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.516187 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.517299 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.517622 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.517664 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.519514 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.519516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.519516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.519523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.521894 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.522313 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.522315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.522315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.522382 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.522605 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.522606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.522606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.522747 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.522828 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.523055 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.523153 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":124,"threads_started":1}
Aug 08 20:56:46 dist-test-slave-c12x krb5kdc[25278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222606, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.188@KRBTEST.COM
I20260808 20:56:46.525491 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.510808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:48267 (local address 127.15.232.190:33823)
0808 20:56:46.510888 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.510890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.510898 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:56:46.512035 (+ 1137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.512037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.512052 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.512100 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.513329 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.514042 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.515776 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.516078 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.516128 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.519162 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.519163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.519163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.519170 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.521930 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.524247 (+ 2317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.524249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.524249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.524317 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.524649 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.524650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.524650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.524794 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.524854 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.525281 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.525375 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":50}
I20260808 20:56:46.526059 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.510307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.131:48267)
0808 20:56:46.510508 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.510554 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.511886 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.512295 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.512300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.512865 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.513203 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.513210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.514177 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.514178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.515680 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.515683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.516222 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.516223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.516366 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.516948 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.516959 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.519044 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.523701 (+ 4657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.523714 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.523717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.524150 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.524403 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.524404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.524404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.524563 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.524893 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.524901 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.525087 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.525590 (+ 503us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.525799 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":24,"thread_start_us":103,"threads_started":1}
I20260808 20:56:46.526736 27225 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33823
I20260808 20:56:46.526850 27225 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:46.526989 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.510764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:38561 (local address 127.15.232.189:38629)
0808 20:56:46.510870 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.510872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.511024 (+ 152us) server_negotiation.cc:407] Connection header received
0808 20:56:46.512045 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.512046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.512059 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.512090 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.513572 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.514422 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.520090 (+ 5668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.520418 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.520467 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.523169 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.523170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.523170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.523181 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.525748 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.526126 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.526127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.526127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.526189 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.526387 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.526388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.526388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.526508 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.526592 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.526824 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.526908 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 20:56:46.527053 27225 heartbeater.cc:511] Master 127.15.232.190:33823 requested a full tablet report, sending...
I20260808 20:56:46.527585 25335 ts_manager.cc:194] Re-registered known tserver with Master: edb4e94b0b374868a5763a1af93e77e2 (127.15.232.131:35453)
I20260808 20:56:46.528076 25335 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:48267
I20260808 20:56:46.525882 25336 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 20:56:46.528275 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.510285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:36393 (local address 127.15.232.188:46165)
0808 20:56:46.510351 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.510353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.510839 (+ 486us) server_negotiation.cc:407] Connection header received
0808 20:56:46.512033 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.512035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.512050 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.512086 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.515774 (+ 3688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.516461 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.517887 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.518588 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.518637 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.524231 (+ 5594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.524232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.524233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.524240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.525969 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.527361 (+ 1392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.527362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.527363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.527426 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.527712 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.527713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.527713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.527835 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.527916 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.528083 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.528174 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260808 20:56:46.528414 25336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:46.528855 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.510682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.15.232.131:38561)
0808 20:56:46.510958 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.510968 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.511899 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.512295 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.512300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.513363 (+ 1063us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.513465 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.513468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.514521 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.514521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.515673 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.515676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.520560 (+ 4884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.520561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.520714 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.521134 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.521139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.523022 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.525840 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.525841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.525841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.526036 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.526268 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.526269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.526269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.526330 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.526602 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.526603 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.526645 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.526712 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.528720 (+ 2008us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260808 20:56:46.529187 27230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.510307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.15.232.131:36393)
0808 20:56:46.510770 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.510781 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.511899 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.512299 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.512300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.512868 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:46.513203 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.513209 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.516555 (+ 3346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.516556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.517792 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.517795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.518818 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.518819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.518903 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.522089 (+ 3186us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:46.522094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:46.524140 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:46.526708 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.526709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.526709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.526936 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.527557 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:46.527557 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:46.527558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:46.527625 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:46.527922 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:46.527923 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:46.527963 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.528044 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.528825 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":645,"thread_start_us":100,"threads_started":1}
I20260808 20:56:46.529242 27227 heartbeater.cc:347] Connected to a master server at 127.15.232.189:38629
I20260808 20:56:46.529294 27227 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:46.529371 27227 heartbeater.cc:511] Master 127.15.232.189:38629 requested a full tablet report, sending...
I20260808 20:56:46.529518 27226 heartbeater.cc:347] Connected to a master server at 127.15.232.188:46165
I20260808 20:56:46.529560 27226 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:46.529606 27226 heartbeater.cc:511] Master 127.15.232.188:46165 requested a full tablet report, sending...
I20260808 20:56:46.529695 26489 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: NON_VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: true } } }
I20260808 20:56:46.529795 26489 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:46.529834 26242 ts_manager.cc:194] Registered new tserver with Master: edb4e94b0b374868a5763a1af93e77e2 (127.15.232.131:35453)
I20260808 20:56:46.529852 26548 ts_manager.cc:194] Registered new tserver with Master: edb4e94b0b374868a5763a1af93e77e2 (127.15.232.131:35453)
I20260808 20:56:46.530714 26489 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: attempting to promote NON_VOTER 75a504a288f84b39a25966125c39c5f6 to VOTER
I20260808 20:56:46.530882 26489 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } }
I20260808 20:56:46.531399 26562 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 LEARNER]: Refusing update from remote peer ea87f99a39bd455392ccdd6f0fe2909e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 20:56:46.531632 26261 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Refusing update from remote peer ea87f99a39bd455392ccdd6f0fe2909e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 20:56:46.531634 26489 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 20:56:46.531869 26489 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [LEADER]: Connected to new peer: Peer: permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 20:56:46.532738 26489 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 75a504a288f84b39a25966125c39c5f6 (127.15.232.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:46.532743 26261 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 75a504a288f84b39a25966125c39c5f6 (127.15.232.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:46.534056 27256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:46.534174 27256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:56:46.534359 26562 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 75a504a288f84b39a25966125c39c5f6 (127.15.232.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:46.534440 27245 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:46.534538 27245 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:46.534463 26489 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:46.534575 26489 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:46.534597 27245 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:46.534651 27245 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:46.535526 26604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:46.535629 26604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:56:46.538462 27237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.513231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46876 (local address 127.15.232.189:38629)
0808 20:56:46.520967 (+ 7736us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.520969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.520978 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:46.521036 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.521037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.522626 (+ 1589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.524480 (+ 1854us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.525176 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.531651 (+ 6475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.532801 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.533100 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.533170 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.534811 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.534812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.534813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.534820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.536736 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.537138 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.537139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.537140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.537210 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.537723 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.537724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.537725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.537885 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.537976 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.538238 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.538341 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7711,"thread_start_us":77,"threads_started":1}
I20260808 20:56:46.542706 27264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.539273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:35453 (local address 127.0.0.1:60504)
0808 20:56:46.539422 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.539456 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.539505 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.539810 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.539811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.539813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:46.539855 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.539857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.540860 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.540861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.541714 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.541715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.542474 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.542475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.542546 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.542551 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.542583 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.542625 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260808 20:56:46.543098 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.539429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.15.232.131:35453)
0808 20:56:46.539642 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.539647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.539657 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:46.539695 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.539697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.539718 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.539779 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:46.539936 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.540768 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.541800 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.542351 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.542860 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.542943 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.542971 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.543015 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":145,"threads_started":1}
I20260808 20:56:46.543679 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.539007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:32911 (local address 127.0.0.1:37268)
0808 20:56:46.539038 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.539050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.539088 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.540094 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.540095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.540096 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:46.540138 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.540139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.541053 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.541054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.542170 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.542173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.543398 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.543399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.543514 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.543518 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.543579 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.543618 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260808 20:56:46.543715 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.539544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37268 (local address 127.15.232.130:32911)
0808 20:56:46.539878 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.539883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.539914 (+ 31us) server_negotiation.cc:407] Connection header received
0808 20:56:46.539969 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.539971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.539986 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.540213 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:46.540279 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.540954 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.542267 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.543308 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.543419 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.543500 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.543593 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.543635 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":87,"threads_started":1}
I20260808 20:56:46.543941 27128 tablet_service.cc:1467] Processing CreateTablet for tablet 9db4307746d64327bf485dfb53f8f9ef (DEFAULT_TABLE table=test-table [id=9bae9e204c24426b977b50edb6751fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:46.544373 27128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9db4307746d64327bf485dfb53f8f9ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:46.544622 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.539074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:33485 (local address 127.0.0.1:49968)
0808 20:56:46.539203 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.539210 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.539239 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.541306 (+ 2067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.541307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.541310 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:46.541354 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.541356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.542513 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.542513 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.543482 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.543485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.544366 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.544367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.544414 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.544417 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.544544 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.544577 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":97,"threads_started":1}
I20260808 20:56:46.544749 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.539527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.15.232.129:33485)
0808 20:56:46.541140 (+ 1613us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.541145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.541154 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:46.541190 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.541192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.541207 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.541288 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:46.541418 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.542418 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.543597 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.544286 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.544390 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.544445 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.544560 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.544601 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1574,"thread_start_us":100,"threads_started":1}
I20260808 20:56:46.545375 26718 tablet_service.cc:1467] Processing CreateTablet for tablet 9db4307746d64327bf485dfb53f8f9ef (DEFAULT_TABLE table=test-table [id=9bae9e204c24426b977b50edb6751fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:46.545989 26718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9db4307746d64327bf485dfb53f8f9ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:46.547567 27268 tablet_bootstrap.cc:492] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: Bootstrap starting.
I20260808 20:56:46.548381 27268 tablet_bootstrap.cc:654] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:46.548473 26915 tablet_service.cc:1467] Processing CreateTablet for tablet 9db4307746d64327bf485dfb53f8f9ef (DEFAULT_TABLE table=test-table [id=9bae9e204c24426b977b50edb6751fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:46.548905 27268 log.cc:824] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:46.549052 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9db4307746d64327bf485dfb53f8f9ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:46.549090 27269 tablet_bootstrap.cc:492] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: Bootstrap starting.
I20260808 20:56:46.549772 27269 tablet_bootstrap.cc:654] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:46.550118 27269 log.cc:824] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:46.550251 27268 tablet_bootstrap.cc:492] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: No bootstrap required, opened a new log
I20260808 20:56:46.550330 27268 ts_tablet_manager.cc:1397] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 20:56:46.551033 27269 tablet_bootstrap.cc:492] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: No bootstrap required, opened a new log
I20260808 20:56:46.551103 27269 ts_tablet_manager.cc:1397] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:46.553318 27268 raft_consensus.cc:348] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } }
I20260808 20:56:46.553380 27269 raft_consensus.cc:348] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } }
I20260808 20:56:46.553503 27268 raft_consensus.cc:374] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:46.553514 27269 raft_consensus.cc:374] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:46.553548 27269 raft_consensus.cc:729] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5288b5f446634ee39afe9dff54506397, State: Initialized, Role: FOLLOWER
I20260808 20:56:46.553548 27268 raft_consensus.cc:729] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb4e94b0b374868a5763a1af93e77e2, State: Initialized, Role: FOLLOWER
I20260808 20:56:46.553673 27269 consensus_queue.cc:260] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } }
I20260808 20:56:46.553709 27268 consensus_queue.cc:260] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } }
I20260808 20:56:46.553968 27269 ts_tablet_manager.cc:1428] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 20:56:46.553974 27268 ts_tablet_manager.cc:1428] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 20:56:46.554092 26812 heartbeater.cc:503] Master 127.15.232.190:33823 was elected leader, sending a full tablet report...
I20260808 20:56:46.553992 27275 tablet_bootstrap.cc:492] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: Bootstrap starting.
I20260808 20:56:46.554483 27225 heartbeater.cc:503] Master 127.15.232.190:33823 was elected leader, sending a full tablet report...
I20260808 20:56:46.556319 27275 tablet_bootstrap.cc:654] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:46.556710 27275 log.cc:824] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:46.557814 27275 tablet_bootstrap.cc:492] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: No bootstrap required, opened a new log
I20260808 20:56:46.557915 27275 ts_tablet_manager.cc:1397] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 20:56:46.560947 27275 raft_consensus.cc:348] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } }
I20260808 20:56:46.561195 27275 raft_consensus.cc:374] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:46.561240 27275 raft_consensus.cc:729] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dd2c1983554bb78e7040f3dfbebf0b, State: Initialized, Role: FOLLOWER
I20260808 20:56:46.561374 27275 consensus_queue.cc:260] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } }
I20260808 20:56:46.561766 27275 ts_tablet_manager.cc:1428] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 20:56:46.561884 27022 heartbeater.cc:503] Master 127.15.232.190:33823 was elected leader, sending a full tablet report...
W20260808 20:56:46.567922 27027 tablet.cc:2369] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 20:56:46.659785 26817 tablet.cc:2369] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 20:56:46.760519 27228 tablet.cc:2369] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:46.781611 27281 raft_consensus.cc:482] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:46.781775 27281 raft_consensus.cc:504] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } }
I20260808 20:56:46.782239 27281 leader_election.cc:288] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5288b5f446634ee39afe9dff54506397 (127.15.232.129:33485), edb4e94b0b374868a5763a1af93e77e2 (127.15.232.131:35453)
I20260808 20:56:46.785338 27265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.782830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:53055 (local address 127.15.232.131:35453)
0808 20:56:46.782884 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.782886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.782893 (+ 7us) server_negotiation.cc:407] Connection header received
0808 20:56:46.782914 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.782915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.782923 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.782994 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:46.783163 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.783663 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.784600 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.784964 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.785052 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.785062 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.785219 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.785260 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 20:56:46.785372 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.782483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:33485 (local address 127.15.232.130:38707)
0808 20:56:46.782551 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.782565 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.782641 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.782810 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.782811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.782813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:46.782902 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.782904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.783711 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.783712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.784494 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.784496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.785054 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.785055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.785155 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.785158 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.785239 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.785283 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":6}
I20260808 20:56:46.785359 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.782614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:38707 (local address 127.15.232.129:33485)
0808 20:56:46.782663 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.782666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.782693 (+ 27us) server_negotiation.cc:407] Connection header received
0808 20:56:46.782730 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.782731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.782740 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.782788 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:46.783015 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.783619 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.784618 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.784970 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.785062 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.785071 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.785248 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.785288 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 20:56:46.785389 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.782608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:35453 (local address 127.15.232.130:53055)
0808 20:56:46.782656 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:46.782666 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:46.782722 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:46.782999 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:46.783000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:46.783002 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:46.783088 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.783090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.783759 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.783760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.784497 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:46.784499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.785052 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.785053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:46.785141 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.785145 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:46.785213 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 20:56:46.785255 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":5}
I20260808 20:56:46.785732 26748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db4307746d64327bf485dfb53f8f9ef" candidate_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5288b5f446634ee39afe9dff54506397" is_pre_election: true
I20260808 20:56:46.785845 27155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db4307746d64327bf485dfb53f8f9ef" candidate_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb4e94b0b374868a5763a1af93e77e2" is_pre_election: true
I20260808 20:56:46.785964 26748 raft_consensus.cc:2454] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dd2c1983554bb78e7040f3dfbebf0b in term 0.
I20260808 20:56:46.786010 27155 raft_consensus.cc:2454] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dd2c1983554bb78e7040f3dfbebf0b in term 0.
I20260808 20:56:46.786221 26850 leader_election.cc:302] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5288b5f446634ee39afe9dff54506397, 99dd2c1983554bb78e7040f3dfbebf0b; no voters:
I20260808 20:56:46.786367 27281 raft_consensus.cc:2781] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:46.786453 27281 raft_consensus.cc:482] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:46.786491 27281 raft_consensus.cc:3037] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:46.787392 27281 raft_consensus.cc:504] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } }
I20260808 20:56:46.787591 27281 leader_election.cc:288] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [CANDIDATE]: Term 1 election: Requested vote from peers 5288b5f446634ee39afe9dff54506397 (127.15.232.129:33485), edb4e94b0b374868a5763a1af93e77e2 (127.15.232.131:35453)
I20260808 20:56:46.787760 26748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db4307746d64327bf485dfb53f8f9ef" candidate_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5288b5f446634ee39afe9dff54506397"
I20260808 20:56:46.787814 27155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9db4307746d64327bf485dfb53f8f9ef" candidate_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb4e94b0b374868a5763a1af93e77e2"
I20260808 20:56:46.787853 26748 raft_consensus.cc:3037] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:46.787890 27155 raft_consensus.cc:3037] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:46.788861 27155 raft_consensus.cc:2454] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dd2c1983554bb78e7040f3dfbebf0b in term 1.
I20260808 20:56:46.788861 26748 raft_consensus.cc:2454] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dd2c1983554bb78e7040f3dfbebf0b in term 1.
I20260808 20:56:46.789067 26850 leader_election.cc:302] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5288b5f446634ee39afe9dff54506397, 99dd2c1983554bb78e7040f3dfbebf0b; no voters:
I20260808 20:56:46.789184 27281 raft_consensus.cc:2781] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:46.789382 27281 raft_consensus.cc:686] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 LEADER]: Becoming Leader. State: Replica: 99dd2c1983554bb78e7040f3dfbebf0b, State: Running, Role: LEADER
I20260808 20:56:46.789489 27281 consensus_queue.cc:237] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } }
I20260808 20:56:46.790437 25335 catalog_manager.cc:5495] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 99dd2c1983554bb78e7040f3dfbebf0b (127.15.232.130). New cstate: current_term: 1 leader_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } health_report { overall_health: UNKNOWN } } }
I20260808 20:56:46.827844 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.825397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37278 (local address 127.15.232.130:32911)
0808 20:56:46.825473 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.825475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.825484 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:46.825532 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.825533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.825552 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.825617 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:46.825746 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.826213 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.827001 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.827195 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.827262 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.827385 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:46.827550 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:46.827615 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.827657 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.827736 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 20:56:46.829829 26748 raft_consensus.cc:1261] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [term 1 FOLLOWER]: Refusing update from remote peer 99dd2c1983554bb78e7040f3dfbebf0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:46.829869 27155 raft_consensus.cc:1261] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [term 1 FOLLOWER]: Refusing update from remote peer 99dd2c1983554bb78e7040f3dfbebf0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:46.830297 27282 consensus_queue.cc:1035] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:46.830438 27281 consensus_queue.cc:1035] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:46.833451 27288 mvcc.cc:205] Tried to move back new op lower bound from 7316367797571833856 to 7316367797410115584. Current Snapshot: MvccSnapshot[applied={T|T < 7316367797571833856}]
I20260808 20:56:46.836290 25335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:46.836385 25335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:46.837667 27290 mvcc.cc:205] Tried to move back new op lower bound from 7316367797571833856 to 7316367797410115584. Current Snapshot: MvccSnapshot[applied={T|T < 7316367797571833856}]
I20260808 20:56:46.838217 25335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea87f99a39bd455392ccdd6f0fe2909e: Sending DeleteTablet for 3 replicas of tablet 9db4307746d64327bf485dfb53f8f9ef
I20260808 20:56:46.838506 26915 tablet_service.cc:1514] Processing DeleteTablet for tablet 9db4307746d64327bf485dfb53f8f9ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:46 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:37268
I20260808 20:56:46.838974 27126 tablet_service.cc:1514] Processing DeleteTablet for tablet 9db4307746d64327bf485dfb53f8f9ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:46 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:60504
I20260808 20:56:46.839043 26718 tablet_service.cc:1514] Processing DeleteTablet for tablet 9db4307746d64327bf485dfb53f8f9ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:46 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:49968
I20260808 20:56:46.838974 27128 tablet_service.cc:1514] Processing DeleteTablet for tablet 9db4307746d64327bf485dfb53f8f9ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:46 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:60504
I20260808 20:56:46.839910 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 25295
I20260808 20:56:46.840072 27305 tablet_replica.cc:330] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: stopping tablet replica
I20260808 20:56:46.840184 27305 raft_consensus.cc:2229] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:46.840289 27305 raft_consensus.cc:2258] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:46.845260 27305 ts_tablet_manager.cc:1905] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:46.847266 27305 ts_tablet_manager.cc:1918] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:46.847385 27305 log.cc:1196] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/9db4307746d64327bf485dfb53f8f9ef
I20260808 20:56:46.847695 27305 ts_tablet_manager.cc:1939] T 9db4307746d64327bf485dfb53f8f9ef P 5288b5f446634ee39afe9dff54506397: Deleting consensus metadata
I20260808 20:56:46.850894 27301 tablet_replica.cc:330] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: stopping tablet replica
I20260808 20:56:46.851070 27301 raft_consensus.cc:2229] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:46.851207 27301 raft_consensus.cc:2258] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:46.851475 27306 tablet_replica.cc:330] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: stopping tablet replica
I20260808 20:56:46.851608 27306 raft_consensus.cc:2229] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:46.851835 27306 raft_consensus.cc:2258] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:46.852478 27301 ts_tablet_manager.cc:1905] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:46.854502 27301 ts_tablet_manager.cc:1918] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:46.854604 27301 log.cc:1196] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/9db4307746d64327bf485dfb53f8f9ef
I20260808 20:56:46.854986 27301 ts_tablet_manager.cc:1939] T 9db4307746d64327bf485dfb53f8f9ef P 99dd2c1983554bb78e7040f3dfbebf0b: Deleting consensus metadata
W20260808 20:56:46.860055 26654 connection.cc:535] server connection from 127.0.0.1:49968 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 20:56:46.860419 26851 connection.cc:535] server connection from 127.0.0.1:37268 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 20:56:46.860517 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.839274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56624 (local address 127.15.232.188:46165)
0808 20:56:46.839319 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.839322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.850677 (+ 11355us) server_negotiation.cc:407] Connection header received
0808 20:56:46.850791 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.850793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.850809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.850864 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.852543 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.853339 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.854773 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.855124 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.855168 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.857221 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.857223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.857223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.857232 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.859003 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.859484 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.859486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.859486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.859556 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.859819 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.859820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.859820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.859980 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.860080 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.860297 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.860391 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:56:46.861457 27306 ts_tablet_manager.cc:1905] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:46.862957 27306 ts_tablet_manager.cc:1918] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:46.863075 27306 log.cc:1196] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/9db4307746d64327bf485dfb53f8f9ef
I20260808 20:56:46.863376 27306 ts_tablet_manager.cc:1939] T 9db4307746d64327bf485dfb53f8f9ef P edb4e94b0b374868a5763a1af93e77e2: Deleting consensus metadata
W20260808 20:56:46.863891 27063 connection.cc:441] server connection from 127.0.0.1:60504 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60504 (request call id 2) could send its response
W20260808 20:56:46.863977 27063 connection.cc:441] server connection from 127.0.0.1:60504 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60504 (request call id 1) could send its response
I20260808 20:56:46.872967 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.864239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.15.232.188:46165)
0808 20:56:46.864282 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.864283 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.864537 (+ 254us) server_negotiation.cc:407] Connection header received
0808 20:56:46.864562 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.864564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.864575 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.864648 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.865577 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.866361 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.867197 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.867405 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.867650 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.869286 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.869287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.869288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.869295 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.871356 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.872028 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.872030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.872030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.872090 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.872320 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.872321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.872321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.872495 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.872603 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.872756 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.872856 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:56:46.873324 27237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:46.863890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.15.232.189:38629)
0808 20:56:46.863977 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:46.863980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:46.864493 (+ 513us) server_negotiation.cc:407] Connection header received
0808 20:56:46.864563 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:46.864565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:46.864582 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:46.864620 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:46.865913 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.866435 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.867540 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:46.867808 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:46.867890 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:46.869158 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:46.869159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:46.869160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:46.869167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:46.871473 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.871997 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.871998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.871999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.872126 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:46.872394 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:46.872395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:46.872395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:46.872538 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:46.872609 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:46.872877 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 20:56:46.873221 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 20:56:47.834892 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:47.834508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.129:36447)
0808 20:56:47.834703 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:47.834805 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":106,"threads_started":1}
W20260808 20:56:47.835371 26812 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:33823 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
I20260808 20:56:47.837517 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:47.837247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.130:59889)
0808 20:56:47.837366 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:47.837469 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
W20260808 20:56:47.837730 27022 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:33823 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
I20260808 20:56:47.840051 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:47.839809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.131:38619)
0808 20:56:47.839935 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:47.840015 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
W20260808 20:56:47.840291 27225 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:33823 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
I20260808 20:56:48.338805 27324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ea87f99a39bd455392ccdd6f0fe2909e)
I20260808 20:56:48.338806 27323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ea87f99a39bd455392ccdd6f0fe2909e)
I20260808 20:56:48.338989 27323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } }
I20260808 20:56:48.338951 27324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } }
I20260808 20:56:48.339321 27323 leader_election.cc:288] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea87f99a39bd455392ccdd6f0fe2909e (127.15.232.190:33823), 98491c8938bf48b6b78207f7f2ba7638 (127.15.232.189:38629)
I20260808 20:56:48.339340 27324 leader_election.cc:288] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ea87f99a39bd455392ccdd6f0fe2909e (127.15.232.190:33823), 75a504a288f84b39a25966125c39c5f6 (127.15.232.188:46165)
I20260808 20:56:48.339751 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.339484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:44140)
0808 20:56:48.339646 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.339701 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":58,"threads_started":1}
I20260808 20:56:48.339820 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.339531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:44156)
0808 20:56:48.339714 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.339771 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":50,"threads_started":1}
W20260808 20:56:48.339951 26227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
W20260808 20:56:48.340010 26533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
I20260808 20:56:48.340415 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.340317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:44168)
0808 20:56:48.340357 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.340387 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 20:56:48.340541 26227 leader_election.cc:334] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ea87f99a39bd455392ccdd6f0fe2909e (127.15.232.190:33823): Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
I20260808 20:56:48.340688 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.340423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:44176)
0808 20:56:48.340634 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.340664 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 20:56:48.340755 26533 leader_election.cc:334] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ea87f99a39bd455392ccdd6f0fe2909e (127.15.232.190:33823): Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
I20260808 20:56:48.342713 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.339546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46165 (local address 127.0.0.1:56642)
0808 20:56:48.339856 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.339866 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:48.339929 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:48.340178 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:48.340179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:48.340182 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:48.340242 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.340245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.341133 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.341135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.341884 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.341886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.342441 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.342442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.342519 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.342522 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:48.342600 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 20:56:48.342644 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":3,"thread_start_us":50,"threads_started":1}
I20260808 20:56:48.342707 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.339746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56642 (local address 127.15.232.188:46165)
0808 20:56:48.339994 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:48.339996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:48.340027 (+ 31us) server_negotiation.cc:407] Connection header received
0808 20:56:48.340050 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:48.340051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:48.340060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:48.340120 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:48.340324 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.341043 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.341986 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.342358 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.342443 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.342453 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:48.342609 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 20:56:48.342650 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":236,"thread_start_us":52,"threads_started":1}
I20260808 20:56:48.343008 26562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98491c8938bf48b6b78207f7f2ba7638" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "75a504a288f84b39a25966125c39c5f6" is_pre_election: true
I20260808 20:56:48.343155 26562 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98491c8938bf48b6b78207f7f2ba7638 in term 1.
I20260808 20:56:48.343266 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.339729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.15.232.189:38629)
0808 20:56:48.339822 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:48.339824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:48.340006 (+ 182us) server_negotiation.cc:407] Connection header received
0808 20:56:48.340032 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:48.340033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:48.340042 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:48.340103 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:48.340546 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.341206 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.342231 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.342646 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.342741 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.342749 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:48.343108 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 20:56:48.343183 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":76,"thread_start_us":50,"threads_started":1}
I20260808 20:56:48.343374 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.339574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:38629 (local address 127.0.0.1:46888)
0808 20:56:48.339838 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.339849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:48.339918 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:48.340169 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:48.340171 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:48.340299 (+ 128us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:48.340439 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.340443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.341300 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.341301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.342129 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.342131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.342740 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.342741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.342996 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.343003 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 20:56:48.343082 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 20:56:48.343133 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":65,"thread_start_us":67,"threads_started":1}
I20260808 20:56:48.343427 26221 leader_election.cc:302] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75a504a288f84b39a25966125c39c5f6, 98491c8938bf48b6b78207f7f2ba7638; no voters: ea87f99a39bd455392ccdd6f0fe2909e
I20260808 20:56:48.343572 27324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 20:56:48.343629 27324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ea87f99a39bd455392ccdd6f0fe2909e)
I20260808 20:56:48.343652 27324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 1 FOLLOWER]: Advancing to term 2
I20260808 20:56:48.343664 26261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75a504a288f84b39a25966125c39c5f6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "98491c8938bf48b6b78207f7f2ba7638" is_pre_election: true
I20260808 20:56:48.344429 27324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } }
I20260808 20:56:48.344584 26261 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75a504a288f84b39a25966125c39c5f6 in current term 2: Already voted for candidate 98491c8938bf48b6b78207f7f2ba7638 in this term.
I20260808 20:56:48.344611 27324 leader_election.cc:288] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [CANDIDATE]: Term 2 election: Requested vote from peers ea87f99a39bd455392ccdd6f0fe2909e (127.15.232.190:33823), 75a504a288f84b39a25966125c39c5f6 (127.15.232.188:46165)
I20260808 20:56:48.344779 26562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98491c8938bf48b6b78207f7f2ba7638" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "75a504a288f84b39a25966125c39c5f6"
I20260808 20:56:48.344822 26528 leader_election.cc:302] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75a504a288f84b39a25966125c39c5f6; no voters: 98491c8938bf48b6b78207f7f2ba7638, ea87f99a39bd455392ccdd6f0fe2909e
I20260808 20:56:48.344900 26562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 1 FOLLOWER]: Advancing to term 2
I20260808 20:56:48.344960 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.344742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:44186)
0808 20:56:48.344881 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.344934 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 20:56:48.345286 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.345172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:44194)
0808 20:56:48.345228 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.345262 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 20:56:48.345373 26227 leader_election.cc:334] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ea87f99a39bd455392ccdd6f0fe2909e (127.15.232.190:33823): Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
I20260808 20:56:48.345777 26562 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98491c8938bf48b6b78207f7f2ba7638 in term 2.
I20260808 20:56:48.345875 27323 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 20:56:48.345995 26221 leader_election.cc:302] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75a504a288f84b39a25966125c39c5f6, 98491c8938bf48b6b78207f7f2ba7638; no voters: ea87f99a39bd455392ccdd6f0fe2909e
I20260808 20:56:48.346107 27324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 20:56:48.346264 27324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [term 2 LEADER]: Becoming Leader. State: Replica: 98491c8938bf48b6b78207f7f2ba7638, State: Running, Role: LEADER
I20260808 20:56:48.346357 27324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } }
I20260808 20:56:48.346796 27338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98491c8938bf48b6b78207f7f2ba7638. Latest consensus state: current_term: 2 leader_uuid: "98491c8938bf48b6b78207f7f2ba7638" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:48.346915 27338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:48.347122 27341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:48.347774 27341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:48.347899 27341 catalog_manager.cc:1259] Loaded cluster ID: 52d888ded2cb4dba8eba7e35c91af6e6
I20260808 20:56:48.347951 27341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:48.348147 27341 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:48.348251 27341 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Loaded TSK: 0
I20260808 20:56:48.348431 27341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 20:56:48.363638 26241 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:48.365348 26241 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:48.366510 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.366400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:44214)
0808 20:56:48.366438 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.366474 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 20:56:48.366640 26562 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [term 2 FOLLOWER]: Refusing update from remote peer 98491c8938bf48b6b78207f7f2ba7638: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 20:56:48.366989 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.366820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.0.0.1:44216)
0808 20:56:48.366898 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.366941 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 20:56:48.366935 27324 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 20:56:48.367130 26227 consensus_peers.cc:487] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 -> Peer ea87f99a39bd455392ccdd6f0fe2909e (127.15.232.190:33823): Couldn't send request to peer ea87f99a39bd455392ccdd6f0fe2909e. Status: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 20:56:48.367730 27323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98491c8938bf48b6b78207f7f2ba7638. Latest consensus state: current_term: 2 leader_uuid: "98491c8938bf48b6b78207f7f2ba7638" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:48.367865 27323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:56:48.368281 27324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "98491c8938bf48b6b78207f7f2ba7638" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:48.368391 27324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:48.368460 27323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "98491c8938bf48b6b78207f7f2ba7638" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "ea87f99a39bd455392ccdd6f0fe2909e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33823 } } peers { permanent_uuid: "98491c8938bf48b6b78207f7f2ba7638" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 38629 } attrs { promote: false } } peers { permanent_uuid: "75a504a288f84b39a25966125c39c5f6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46165 } attrs { promote: false } } }
I20260808 20:56:48.368542 27323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a504a288f84b39a25966125c39c5f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:56:48.374277 27355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.370673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.15.232.130:32911)
0808 20:56:48.371132 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:48.371134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:48.371140 (+ 6us) server_negotiation.cc:407] Connection header received
0808 20:56:48.371164 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:48.371165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:48.371175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:48.371235 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:48.371369 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.372127 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.373313 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.373880 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.373993 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.374002 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:48.374160 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 20:56:48.374201 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":75,"threads_started":1}
I20260808 20:56:48.374279 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.370556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:35453 (local address 127.0.0.1:60512)
0808 20:56:48.370665 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.370678 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:48.370737 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:48.370910 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:48.370912 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:48.370916 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:48.370981 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.370983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.371995 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.371996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.373254 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.373257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.374013 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.374014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.374090 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.374093 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:48.374139 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 20:56:48.374190 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":30}
I20260808 20:56:48.374439 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.370582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:32911 (local address 127.0.0.1:37288)
0808 20:56:48.370972 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.370983 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:48.371027 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:48.371242 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:48.371243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:48.371245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:48.371289 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.371290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.372205 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.372206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.373223 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.373225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.373993 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.373995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.374077 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.374086 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 20:56:48.374135 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 20:56:48.374171 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":3}
I20260808 20:56:48.374595 27352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.370639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.15.232.131:35453)
0808 20:56:48.370769 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:48.370771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:48.370779 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:56:48.370805 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:48.370806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:48.370816 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:48.370883 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:48.371057 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.371910 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.373351 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.373909 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.374416 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.374427 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:48.374446 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 20:56:48.374492 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260808 20:56:48.374722 26915 tablet_service.cc:1467] Processing CreateTablet for tablet 888da3bb0a194824b179a63fdad3a5ca (DEFAULT_TABLE table=test-table [id=f4d1cffd178e499ba6283f921582b63e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:48.374828 27128 tablet_service.cc:1467] Processing CreateTablet for tablet 888da3bb0a194824b179a63fdad3a5ca (DEFAULT_TABLE table=test-table [id=f4d1cffd178e499ba6283f921582b63e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:48.374881 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 888da3bb0a194824b179a63fdad3a5ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:48.374938 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.370648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:33485 (local address 127.0.0.1:49976)
0808 20:56:48.371132 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.371143 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:48.371205 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:48.371425 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:48.371425 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:48.371427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:48.371480 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.371481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.372364 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.372365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.373400 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:48.373402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.374215 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.374216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:48.374298 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.374302 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:56:48.374818 (+ 516us) client_negotiation.cc:240] Negotiation successful
0808 20:56:48.374871 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":104,"threads_started":1}
I20260808 20:56:48.374984 27128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 888da3bb0a194824b179a63fdad3a5ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:48.375021 27356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.370865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49976 (local address 127.15.232.129:33485)
0808 20:56:48.370999 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:48.371001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:48.371265 (+ 264us) server_negotiation.cc:407] Connection header received
0808 20:56:48.371299 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:48.371300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:48.371312 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:48.371376 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:48.371558 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.372247 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.373473 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.373879 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.373988 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.373999 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:48.374846 (+ 847us) server_negotiation.cc:299] Negotiation successful
0808 20:56:48.374919 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260808 20:56:48.376194 27357 tablet_bootstrap.cc:492] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: Bootstrap starting.
I20260808 20:56:48.376470 27359 tablet_bootstrap.cc:492] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: Bootstrap starting.
I20260808 20:56:48.376556 26718 tablet_service.cc:1467] Processing CreateTablet for tablet 888da3bb0a194824b179a63fdad3a5ca (DEFAULT_TABLE table=test-table [id=f4d1cffd178e499ba6283f921582b63e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:48.376670 27357 tablet_bootstrap.cc:654] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:48.376708 26718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 888da3bb0a194824b179a63fdad3a5ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:48.376979 27359 tablet_bootstrap.cc:654] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:48.377617 27357 tablet_bootstrap.cc:492] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: No bootstrap required, opened a new log
I20260808 20:56:48.377658 27357 ts_tablet_manager.cc:1397] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:48.377693 27359 tablet_bootstrap.cc:492] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: No bootstrap required, opened a new log
I20260808 20:56:48.377743 27359 ts_tablet_manager.cc:1397] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:48.377743 27357 raft_consensus.cc:348] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } }
I20260808 20:56:48.377790 27357 raft_consensus.cc:374] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:48.377804 27357 raft_consensus.cc:729] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dd2c1983554bb78e7040f3dfbebf0b, State: Initialized, Role: FOLLOWER
I20260808 20:56:48.377863 27357 consensus_queue.cc:260] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } }
I20260808 20:56:48.377894 27359 raft_consensus.cc:348] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } }
I20260808 20:56:48.377974 27359 raft_consensus.cc:374] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:48.378001 27359 raft_consensus.cc:729] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb4e94b0b374868a5763a1af93e77e2, State: Initialized, Role: FOLLOWER
I20260808 20:56:48.378023 27357 ts_tablet_manager.cc:1428] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:48.378055 27359 consensus_queue.cc:260] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } }
I20260808 20:56:48.378230 27359 ts_tablet_manager.cc:1428] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:48.378770 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.378225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.130:37077)
0808 20:56:48.378379 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.378727 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260808 20:56:48.378854 27026 heartbeater.cc:503] Master 127.15.232.189:38629 was elected leader, sending a full tablet report...
I20260808 20:56:48.379056 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.378980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.130:36709)
0808 20:56:48.379012 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.379036 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260808 20:56:48.379136 27227 heartbeater.cc:503] Master 127.15.232.189:38629 was elected leader, sending a full tablet report...
I20260808 20:56:48.379166 27363 tablet_bootstrap.cc:492] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397: Bootstrap starting.
I20260808 20:56:48.379623 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.378464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.131:59667)
0808 20:56:48.379541 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.379582 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1034,"thread_start_us":90,"threads_started":1}
I20260808 20:56:48.379897 27363 tablet_bootstrap.cc:654] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:48.380033 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.379820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.131:56659)
0808 20:56:48.379958 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.380000 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":91,"threads_started":1}
I20260808 20:56:48.380578 27363 tablet_bootstrap.cc:492] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397: No bootstrap required, opened a new log
I20260808 20:56:48.380626 27363 ts_tablet_manager.cc:1397] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 20:56:48.380743 27363 raft_consensus.cc:348] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } }
I20260808 20:56:48.380796 27363 raft_consensus.cc:374] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:48.380810 27363 raft_consensus.cc:729] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5288b5f446634ee39afe9dff54506397, State: Initialized, Role: FOLLOWER
I20260808 20:56:48.380846 27363 consensus_queue.cc:260] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } }
I20260808 20:56:48.380976 27363 ts_tablet_manager.cc:1428] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:48.381454 26814 heartbeater.cc:503] Master 127.15.232.189:38629 was elected leader, sending a full tablet report...
I20260808 20:56:48.381573 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.381166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.129:58921)
0808 20:56:48.381307 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.381347 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":90,"threads_started":1}
I20260808 20:56:48.381978 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.381870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.129:48395)
0808 20:56:48.381908 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.381934 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 20:56:48.398435 27362 raft_consensus.cc:482] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:48.398550 27362 raft_consensus.cc:504] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } }
I20260808 20:56:48.398762 27362 leader_election.cc:288] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edb4e94b0b374868a5763a1af93e77e2 (127.15.232.131:35453), 5288b5f446634ee39afe9dff54506397 (127.15.232.129:33485)
I20260808 20:56:48.398972 27155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "888da3bb0a194824b179a63fdad3a5ca" candidate_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb4e94b0b374868a5763a1af93e77e2" is_pre_election: true
I20260808 20:56:48.399014 26748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "888da3bb0a194824b179a63fdad3a5ca" candidate_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5288b5f446634ee39afe9dff54506397" is_pre_election: true
I20260808 20:56:48.399094 27155 raft_consensus.cc:2454] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dd2c1983554bb78e7040f3dfbebf0b in term 0.
I20260808 20:56:48.399106 26748 raft_consensus.cc:2454] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99dd2c1983554bb78e7040f3dfbebf0b in term 0.
I20260808 20:56:48.399253 26850 leader_election.cc:302] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dd2c1983554bb78e7040f3dfbebf0b, edb4e94b0b374868a5763a1af93e77e2; no voters:
I20260808 20:56:48.399363 27362 raft_consensus.cc:2781] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:48.399417 27362 raft_consensus.cc:482] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:48.399467 27362 raft_consensus.cc:3037] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:48.400317 27362 raft_consensus.cc:504] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } }
I20260808 20:56:48.400460 27362 leader_election.cc:288] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [CANDIDATE]: Term 1 election: Requested vote from peers edb4e94b0b374868a5763a1af93e77e2 (127.15.232.131:35453), 5288b5f446634ee39afe9dff54506397 (127.15.232.129:33485)
I20260808 20:56:48.400636 27155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "888da3bb0a194824b179a63fdad3a5ca" candidate_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edb4e94b0b374868a5763a1af93e77e2"
I20260808 20:56:48.400684 26748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "888da3bb0a194824b179a63fdad3a5ca" candidate_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5288b5f446634ee39afe9dff54506397"
I20260808 20:56:48.400730 27155 raft_consensus.cc:3037] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:48.400750 26748 raft_consensus.cc:3037] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:48.401566 27155 raft_consensus.cc:2454] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dd2c1983554bb78e7040f3dfbebf0b in term 1.
I20260808 20:56:48.401618 26748 raft_consensus.cc:2454] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99dd2c1983554bb78e7040f3dfbebf0b in term 1.
I20260808 20:56:48.401815 26850 leader_election.cc:302] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99dd2c1983554bb78e7040f3dfbebf0b, edb4e94b0b374868a5763a1af93e77e2; no voters:
I20260808 20:56:48.401959 27362 raft_consensus.cc:2781] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:48.402190 27362 raft_consensus.cc:686] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 LEADER]: Becoming Leader. State: Replica: 99dd2c1983554bb78e7040f3dfbebf0b, State: Running, Role: LEADER
I20260808 20:56:48.402244 27362 consensus_queue.cc:237] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } }
I20260808 20:56:48.402573 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.402446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.130:51815)
0808 20:56:48.402503 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.402538 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 20:56:48.403002 26242 catalog_manager.cc:5495] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 99dd2c1983554bb78e7040f3dfbebf0b (127.15.232.130). New cstate: current_term: 1 leader_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99dd2c1983554bb78e7040f3dfbebf0b" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32911 } health_report { overall_health: HEALTHY } } }
I20260808 20:56:48.410043 27355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.407716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.15.232.130:32911)
0808 20:56:48.407804 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:48.407806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:48.407815 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:48.407836 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:48.407837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:48.407846 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:48.407914 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:48.408100 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.408603 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.409351 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:48.409560 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:48.409616 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:48.409791 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:48.409822 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:48.409883 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:48.409967 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 20:56:48.409988 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 20:56:48.410945 26748 raft_consensus.cc:1261] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [term 1 FOLLOWER]: Refusing update from remote peer 99dd2c1983554bb78e7040f3dfbebf0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:48.410964 27155 raft_consensus.cc:1261] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [term 1 FOLLOWER]: Refusing update from remote peer 99dd2c1983554bb78e7040f3dfbebf0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:48.411226 27375 consensus_queue.cc:1035] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5288b5f446634ee39afe9dff54506397" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 33485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:48.411345 27362 consensus_queue.cc:1035] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "edb4e94b0b374868a5763a1af93e77e2" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 35453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:48.412369 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.412256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.129:40847)
0808 20:56:48.412302 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.412332 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 20:56:48.412709 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.412606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.131:51653)
0808 20:56:48.412651 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.412677 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 20:56:48.413028 27381 mvcc.cc:205] Tried to move back new op lower bound from 7316367804049506304 to 7316367804016160768. Current Snapshot: MvccSnapshot[applied={T|T < 7316367804049506304}]
I20260808 20:56:48.413239 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.413136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.130:54341)
0808 20:56:48.413175 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.413210 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 20:56:48.414029 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.413932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.129:50021)
0808 20:56:48.413978 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.414003 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 20:56:48.414664 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.414534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.131:47447)
0808 20:56:48.414599 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.414632 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 20:56:48.414718 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.414633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33823 (local address 127.15.232.130:41417)
0808 20:56:48.414669 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.414692 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:33823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 20:56:48.414814 26242 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:48.414903 26242 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:48.416244 26242 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98491c8938bf48b6b78207f7f2ba7638: Sending DeleteTablet for 3 replicas of tablet 888da3bb0a194824b179a63fdad3a5ca
I20260808 20:56:48.416610 26915 tablet_service.cc:1514] Processing DeleteTablet for tablet 888da3bb0a194824b179a63fdad3a5ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:48 UTC) from {username='oryx', principal='oryx/127.15.232.189@KRBTEST.COM'} at 127.0.0.1:37288
I20260808 20:56:48.416610 26718 tablet_service.cc:1514] Processing DeleteTablet for tablet 888da3bb0a194824b179a63fdad3a5ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:48 UTC) from {username='oryx', principal='oryx/127.15.232.189@KRBTEST.COM'} at 127.0.0.1:49976
I20260808 20:56:48.416633 27128 tablet_service.cc:1514] Processing DeleteTablet for tablet 888da3bb0a194824b179a63fdad3a5ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:48 UTC) from {username='oryx', principal='oryx/127.15.232.189@KRBTEST.COM'} at 127.0.0.1:60512
I20260808 20:56:48.416864 27393 tablet_replica.cc:330] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397: stopping tablet replica
I20260808 20:56:48.416923 27393 raft_consensus.cc:2229] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:48.416988 27393 raft_consensus.cc:2258] T 888da3bb0a194824b179a63fdad3a5ca P 5288b5f446634ee39afe9dff54506397 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:48.417003 27395 tablet_replica.cc:330] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: stopping tablet replica
I20260808 20:56:48.417055 27395 raft_consensus.cc:2229] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:48.417086 27394 tablet_replica.cc:330] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: stopping tablet replica
I20260808 20:56:48.417107 27395 raft_consensus.cc:2258] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:48.417132 27394 raft_consensus.cc:2229] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:48.417176 27394 raft_consensus.cc:2258] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:48.417244 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 26477
I20260808 20:56:48.419842 27394 ts_tablet_manager.cc:1905] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:48.421928 27394 ts_tablet_manager.cc:1918] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:48.421977 27394 log.cc:1196] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/888da3bb0a194824b179a63fdad3a5ca
I20260808 20:56:48.422235 27394 ts_tablet_manager.cc:1939] T 888da3bb0a194824b179a63fdad3a5ca P edb4e94b0b374868a5763a1af93e77e2: Deleting consensus metadata
I20260808 20:56:48.422549 26221 catalog_manager.cc:4841] TS edb4e94b0b374868a5763a1af93e77e2 (127.15.232.131:35453): tablet 888da3bb0a194824b179a63fdad3a5ca (table test-table [id=f4d1cffd178e499ba6283f921582b63e]) successfully deleted
I20260808 20:56:48.427541 27395 ts_tablet_manager.cc:1905] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:48.429617 27395 ts_tablet_manager.cc:1918] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:48.429702 27395 log.cc:1196] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/888da3bb0a194824b179a63fdad3a5ca
I20260808 20:56:48.429963 27395 ts_tablet_manager.cc:1939] T 888da3bb0a194824b179a63fdad3a5ca P 99dd2c1983554bb78e7040f3dfbebf0b: Deleting consensus metadata
I20260808 20:56:48.430369 26226 catalog_manager.cc:4841] TS 99dd2c1983554bb78e7040f3dfbebf0b (127.15.232.130:32911): tablet 888da3bb0a194824b179a63fdad3a5ca (table test-table [id=f4d1cffd178e499ba6283f921582b63e]) successfully deleted
I20260808 20:56:48.432261 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:48.432074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:33485 (local address 127.0.0.1:49988)
0808 20:56:48.432151 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:48.432215 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:33485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 20:56:48.432313 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 26822
W20260808 20:56:48.432487 26221 catalog_manager.cc:4584] TS 5288b5f446634ee39afe9dff54506397 (127.15.232.129:33485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 888da3bb0a194824b179a63fdad3a5ca: Network error: Client connection negotiation failed: client connection to 127.15.232.129:33485: connect: Connection refused (error 111)
I20260808 20:56:48.448288 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 27037
I20260808 20:56:48.457379 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 26038
I20260808 20:56:48.468185 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 26338
2026-08-08T20:56:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5182 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:48 dist-test-slave-c12x krb5kdc[27398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:48 dist-test-slave-c12x krb5kdc[27398](info): set up 2 sockets
Aug 08 20:56:48 dist-test-slave-c12x krb5kdc[27398](info): commencing operation
krb5kdc: starting...
W20260808 20:56:49.695179 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:49 dist-test-slave-c12x krb5kdc[27398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.190.keytab.
Entry for principal oryx/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.190.keytab.
I20260808 20:56:49.826846 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:43705
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.190.keytab
--principal=oryx/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:43705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:49.913170 27418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:49.913416 27418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:49.913452 27418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:49.914599 27418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:49.914667 27418 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 20:56:49.914692 27418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:49.914722 27418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:49.914752 27418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:49.914774 27418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:49.916363 27418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:43705
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.190.keytab
--principal=oryx/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:43705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:49.916646 27418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:49.916875 27418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:49.922199 27428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:49.922260 27430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:49.922212 27427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:49.922307 27418 server_base.cc:1034] running on GCE node
I20260808 20:56:49.922717 27418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:49.923022 27418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:49.924199 27418 hybrid_clock.cc:648] HybridClock initialized: now 1786222609924181 us; error 29 us; skew 500 ppm
Aug 08 20:56:49 dist-test-slave-c12x krb5kdc[27398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222609, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:49.926851 27418 init.cc:377] Logged in from keytab as oryx/127.15.232.190@KRBTEST.COM (short username oryx)
I20260808 20:56:49.927330 27418 webserver.cc:460] Webserver started at http://127.15.232.190:39445/ using document root <none> and password file <none>
I20260808 20:56:49.927584 27418 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:49.927634 27418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:49.927767 27418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:49.928869 27418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "8466e036ad72419486c0b240afd5c49a"
format_stamp: "Formatted at 2026-08-08 20:56:49 on dist-test-slave-c12x"
server_key: "0e0c4829a87fc9babf5dbbbbebb63c79"
server_key_iv: "634665f1a373a21ddf1ed8f75f31c0bc"
server_key_version: "encryptionkey@0"
I20260808 20:56:49.929270 27418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "8466e036ad72419486c0b240afd5c49a"
format_stamp: "Formatted at 2026-08-08 20:56:49 on dist-test-slave-c12x"
server_key: "0e0c4829a87fc9babf5dbbbbebb63c79"
server_key_iv: "634665f1a373a21ddf1ed8f75f31c0bc"
server_key_version: "encryptionkey@0"
I20260808 20:56:49.930636 27418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:49.931371 27442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:49.931546 27418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:49.931663 27418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "8466e036ad72419486c0b240afd5c49a"
format_stamp: "Formatted at 2026-08-08 20:56:49 on dist-test-slave-c12x"
server_key: "0e0c4829a87fc9babf5dbbbbebb63c79"
server_key_iv: "634665f1a373a21ddf1ed8f75f31c0bc"
server_key_version: "encryptionkey@0"
I20260808 20:56:49.931743 27418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:49.956269 27418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:49.958231 27418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:49.958413 27418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:49.963495 27418 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:43705
I20260808 20:56:49.963578 27532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:43705 every 8 connection(s)
I20260808 20:56:49.963918 27418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:49.964679 27533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:49.966982 27533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a: Bootstrap starting.
I20260808 20:56:49.967569 27533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:49.967814 27533 log.cc:824] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:49.968477 27533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a: No bootstrap required, opened a new log
I20260808 20:56:49.970484 27533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8466e036ad72419486c0b240afd5c49a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 43705 } }
I20260808 20:56:49.970623 27533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:49.970644 27533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8466e036ad72419486c0b240afd5c49a, State: Initialized, Role: FOLLOWER
I20260808 20:56:49.970798 27533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8466e036ad72419486c0b240afd5c49a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 43705 } }
I20260808 20:56:49.970888 27533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:49.970912 27533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:49.970940 27533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:49.971491 27533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8466e036ad72419486c0b240afd5c49a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 43705 } }
I20260808 20:56:49.971591 27533 leader_election.cc:302] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8466e036ad72419486c0b240afd5c49a; no voters:
I20260808 20:56:49.971719 27533 leader_election.cc:288] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:49.971843 27537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:49.972018 27537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [term 1 LEADER]: Becoming Leader. State: Replica: 8466e036ad72419486c0b240afd5c49a, State: Running, Role: LEADER
I20260808 20:56:49.972096 27533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:49.972110 27537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8466e036ad72419486c0b240afd5c49a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 43705 } }
I20260808 20:56:49.972520 27540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8466e036ad72419486c0b240afd5c49a. Latest consensus state: current_term: 1 leader_uuid: "8466e036ad72419486c0b240afd5c49a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8466e036ad72419486c0b240afd5c49a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 43705 } } }
I20260808 20:56:49.972497 27538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8466e036ad72419486c0b240afd5c49a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8466e036ad72419486c0b240afd5c49a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 43705 } } }
I20260808 20:56:49.972597 27540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:49.972605 27538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:49.972846 27545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:49.973487 27545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:49.973593 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 27418
I20260808 20:56:49.973714 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:49.973901 16290 external_mini_cluster.cc:1428] Setting key 242662038255e39095779191c19c1653
I20260808 20:56:49.976147 27545 catalog_manager.cc:1347] Generated new cluster ID: 7d68796dd88948d289cdb1ae20464df0
I20260808 20:56:49.976205 27545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 20:56:49 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:49.986157 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:49.974511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.15.232.190:43705)
0808 20:56:49.974669 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:49.974679 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:49.974714 (+ 35us) server_negotiation.cc:407] Connection header received
0808 20:56:49.974849 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:49.974856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:49.974912 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:49.975057 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:49.975827 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:49.977310 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:49.978346 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:49.979622 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:49.980017 (+ 395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:49.982548 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:49.982552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:49.982553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:49.982564 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:49.984524 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:49.984932 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:49.984935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:49.984935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:49.985041 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:49.985247 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:49.985248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:49.985248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:49.985536 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:49.985655 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:49.985795 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 20:56:49.985923 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260808 20:56:50.015211 27545 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:50.015827 27545 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:50.020704 27545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a: Generated new TSK 0
I20260808 20:56:50.020917 27545 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab.
Entry for principal oryx/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab.
I20260808 20:56:50.054339 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:43705
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab
--principal=oryx/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:50.139699 27571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:50.139899 27571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:50.139922 27571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:50.141067 27571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:50.141099 27571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:50.141119 27571 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 20:56:50.141137 27571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:50.142601 27571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab
--principal=oryx/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.190:43705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:50.142789 27571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:50.142957 27571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:50.148247 27581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:50.148247 27583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:50.148300 27585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:50.148609 27571 server_base.cc:1034] running on GCE node
I20260808 20:56:50.148855 27571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:50.149137 27571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:50.150285 27571 hybrid_clock.cc:648] HybridClock initialized: now 1786222610150274 us; error 33 us; skew 500 ppm
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222610, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:50.152999 27571 init.cc:377] Logged in from keytab as oryx/127.15.232.129@KRBTEST.COM (short username oryx)
I20260808 20:56:50.153383 27571 webserver.cc:460] Webserver started at http://127.15.232.129:43375/ using document root <none> and password file <none>
I20260808 20:56:50.153578 27571 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:50.153618 27571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:50.153707 27571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:50.154713 27571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "48c5b28b4acc4cb58e89e035db0cacfe"
format_stamp: "Formatted at 2026-08-08 20:56:50 on dist-test-slave-c12x"
server_key: "85f1025e0f2650687ad33c92e4715a57"
server_key_iv: "3ab1b8c076c8cdfc1b54604b0d2b90e3"
server_key_version: "encryptionkey@0"
I20260808 20:56:50.155014 27571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "48c5b28b4acc4cb58e89e035db0cacfe"
format_stamp: "Formatted at 2026-08-08 20:56:50 on dist-test-slave-c12x"
server_key: "85f1025e0f2650687ad33c92e4715a57"
server_key_iv: "3ab1b8c076c8cdfc1b54604b0d2b90e3"
server_key_version: "encryptionkey@0"
I20260808 20:56:50.156378 27571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:50.157094 27594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:50.157270 27571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 20:56:50.157378 27571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "48c5b28b4acc4cb58e89e035db0cacfe"
format_stamp: "Formatted at 2026-08-08 20:56:50 on dist-test-slave-c12x"
server_key: "85f1025e0f2650687ad33c92e4715a57"
server_key_iv: "3ab1b8c076c8cdfc1b54604b0d2b90e3"
server_key_version: "encryptionkey@0"
I20260808 20:56:50.157459 27571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:50.181715 27571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:50.183789 27571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:50.183959 27571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:50.184520 27571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:50.184561 27571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:50.184624 27571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:50.184651 27571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222610, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:50.191650 27571 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:37431
I20260808 20:56:50.191711 27767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:37431 every 8 connection(s)
I20260808 20:56:50.192116 27571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:50.194927 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.185097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:47529 (local address 127.15.232.190:43705)
0808 20:56:50.185176 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.185179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.185378 (+ 199us) server_negotiation.cc:407] Connection header received
0808 20:56:50.185877 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.185878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.185897 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.185979 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:50.186870 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.187763 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.188896 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.189125 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.189203 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.191899 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:50.191901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:50.191901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:50.191909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:50.193461 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.193892 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.193893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.193893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.193956 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.194234 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.194235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.194236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.194357 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:50.194429 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.194730 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.194833 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 20:56:50.195092 27608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.185020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:43705 (local address 127.15.232.129:47529)
0808 20:56:50.185267 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.185295 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.185774 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.186038 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.186042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.186443 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:50.186754 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.186761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.187864 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.187865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.188785 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.188788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.189228 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.189229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.189384 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.189990 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:50.190000 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:50.191810 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:50.193573 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.193581 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.193583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.193809 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.194038 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.194039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.194040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.194157 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.194435 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:50.194443 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:50.194590 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.194696 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.194861 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":80,"threads_started":1}
I20260808 20:56:50.195633 27768 heartbeater.cc:347] Connected to a master server at 127.15.232.190:43705
I20260808 20:56:50.195741 27768 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:50.195930 27768 heartbeater.cc:511] Master 127.15.232.190:43705 requested a full tablet report, sending...
I20260808 20:56:50.196544 27461 ts_manager.cc:194] Registered new tserver with Master: 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431)
I20260808 20:56:50.197224 27461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:47529
I20260808 20:56:50.200541 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 27571
I20260808 20:56:50.200642 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:50.200937 16290 external_mini_cluster.cc:1428] Setting key afdb2874250c7a4250f916b8ce5b707d
I20260808 20:56:50.202716 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.196649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:43705 (local address 127.15.232.129:60809)
0808 20:56:50.196793 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.196809 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.196881 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.197009 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.197010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.197178 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:50.197231 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.197232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.197915 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.197916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.198417 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.198418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.198787 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.198788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.198860 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.199325 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:50.199335 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:50.199679 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:50.201577 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.201578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.201579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.201775 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.201982 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.201983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.201983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.202067 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.202299 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:50.202300 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:50.202352 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.202431 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.202558 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260808 20:56:50.202826 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.196713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:60809 (local address 127.15.232.190:43705)
0808 20:56:50.196754 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.196756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.196865 (+ 109us) server_negotiation.cc:407] Connection header received
0808 20:56:50.196938 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.196939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.196949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.196997 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:50.197306 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.197817 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.198507 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.198713 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.198789 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.199783 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:50.199784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:50.199784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:50.199791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:50.201473 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.201848 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.201849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.201849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.201909 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.202131 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.202131 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.202132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.202236 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:50.202288 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.202614 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.202711 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab.
Entry for principal oryx/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab.
I20260808 20:56:50.229887 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:43705
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab
--principal=oryx/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:50.314711 27778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:50.314936 27778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:50.314962 27778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:50.316293 27778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:50.316337 27778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:50.316359 27778 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 20:56:50.316377 27778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:50.317899 27778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab
--principal=oryx/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.190:43705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:50.318100 27778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:50.318284 27778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:50.323868 27785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:50.323869 27786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:50.323901 27778 server_base.cc:1034] running on GCE node
W20260808 20:56:50.323869 27790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:50.324424 27778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:50.324699 27778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:50.325878 27778 hybrid_clock.cc:648] HybridClock initialized: now 1786222610325872 us; error 38 us; skew 500 ppm
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222610, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:50.328584 27778 init.cc:377] Logged in from keytab as oryx/127.15.232.130@KRBTEST.COM (short username oryx)
I20260808 20:56:50.329145 27778 webserver.cc:460] Webserver started at http://127.15.232.130:36129/ using document root <none> and password file <none>
I20260808 20:56:50.329413 27778 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:50.329487 27778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:50.329653 27778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:50.330788 27778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c"
format_stamp: "Formatted at 2026-08-08 20:56:50 on dist-test-slave-c12x"
server_key: "aa95d0ce25a6e1f2dca4b598ef66c24d"
server_key_iv: "2034d91e1e39188ca3d34b6d02a83d2b"
server_key_version: "encryptionkey@0"
I20260808 20:56:50.331187 27778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c"
format_stamp: "Formatted at 2026-08-08 20:56:50 on dist-test-slave-c12x"
server_key: "aa95d0ce25a6e1f2dca4b598ef66c24d"
server_key_iv: "2034d91e1e39188ca3d34b6d02a83d2b"
server_key_version: "encryptionkey@0"
I20260808 20:56:50.332698 27778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 20:56:50.333556 27798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:50.333781 27778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:50.333889 27778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c"
format_stamp: "Formatted at 2026-08-08 20:56:50 on dist-test-slave-c12x"
server_key: "aa95d0ce25a6e1f2dca4b598ef66c24d"
server_key_iv: "2034d91e1e39188ca3d34b6d02a83d2b"
server_key_version: "encryptionkey@0"
I20260808 20:56:50.333986 27778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:50.358820 27778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:50.361116 27778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:50.361308 27778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:50.361897 27778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:50.361941 27778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:50.361965 27778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:50.361979 27778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222610, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:50.369599 27778 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:45279
I20260808 20:56:50.369732 27958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:45279 every 8 connection(s)
I20260808 20:56:50.370038 27778 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:50.371994 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.362359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:34849 (local address 127.15.232.190:43705)
0808 20:56:50.362424 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.362427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.362637 (+ 210us) server_negotiation.cc:407] Connection header received
0808 20:56:50.363275 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.363277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.363300 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.363373 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:50.364107 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.364944 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.366042 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.366388 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.366473 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.368937 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:50.368938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:50.368939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:50.368949 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:50.370701 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.371091 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.371093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.371093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.371147 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.371331 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.371331 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.371332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.371484 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:50.371549 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.371813 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.371880 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 20:56:50.372112 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.362328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:43705 (local address 127.15.232.130:34849)
0808 20:56:50.362505 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.362542 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.363112 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.363499 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.363504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.363774 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:50.363980 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.363984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.365060 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.365062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.365959 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.365962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.366494 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.366496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.366626 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.367206 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:50.367215 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:50.368821 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:50.370763 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.370768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.370770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.371006 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.371190 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.371191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.371191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.371279 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.371556 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:50.371562 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:50.371695 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.371781 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.371909 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260808 20:56:50.372611 27959 heartbeater.cc:347] Connected to a master server at 127.15.232.190:43705
I20260808 20:56:50.372695 27959 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:50.372912 27959 heartbeater.cc:511] Master 127.15.232.190:43705 requested a full tablet report, sending...
I20260808 20:56:50.373314 27462 ts_manager.cc:194] Registered new tserver with Master: e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130:45279)
I20260808 20:56:50.373817 27462 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:34849
I20260808 20:56:50.376228 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 27778
I20260808 20:56:50.376377 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:50.376592 16290 external_mini_cluster.cc:1428] Setting key 80bffae40f8ccbd8f68e9fb2c54ce867
I20260808 20:56:50.379647 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.373718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:37501 (local address 127.15.232.190:43705)
0808 20:56:50.373758 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.373760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.373888 (+ 128us) server_negotiation.cc:407] Connection header received
0808 20:56:50.373950 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.373950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.373961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.374010 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:50.374401 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.374880 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.375655 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.375856 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.375939 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.376904 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:50.376905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:50.376906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:50.376913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:50.378301 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.378694 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.378695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.378696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.378759 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.379018 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.379019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.379019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.379238 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:50.379300 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.379508 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.379588 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 20:56:50.379737 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.373678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:43705 (local address 127.15.232.130:37501)
0808 20:56:50.373812 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.373830 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.373901 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.374051 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.374052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.374260 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:50.374310 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.374311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.374977 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.374978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.375561 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.375562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.375943 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.375944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.376009 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.376472 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:50.376477 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:50.376821 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:50.378422 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.378423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.378423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.378605 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.378837 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.378838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.378838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.378923 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.379314 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:50.379315 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:50.379368 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.379470 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.379596 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for oryx/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab.
Entry for principal oryx/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab.
I20260808 20:56:50.404122 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:43705
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab
--principal=oryx/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:50.488960 27971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:50.489187 27971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:50.489213 27971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:50.490327 27971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:50.490361 27971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:50.490382 27971 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 20:56:50.490401 27971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:50.491936 27971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab
--principal=oryx/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.232.190:43705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:50.492130 27971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:50.492301 27971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:50.497785 27980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:50.497855 27977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:50.497785 27978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:50.498076 27971 server_base.cc:1034] running on GCE node
I20260808 20:56:50.498292 27971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:50.498492 27971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:50.499636 27971 hybrid_clock.cc:648] HybridClock initialized: now 1786222610499623 us; error 29 us; skew 500 ppm
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222610, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:50.502406 27971 init.cc:377] Logged in from keytab as oryx/127.15.232.131@KRBTEST.COM (short username oryx)
I20260808 20:56:50.502919 27971 webserver.cc:460] Webserver started at http://127.15.232.131:34959/ using document root <none> and password file <none>
I20260808 20:56:50.503170 27971 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:50.503244 27971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:50.503401 27971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:50.504504 27971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "64d06fbc23484fab9905b6e5172142de"
format_stamp: "Formatted at 2026-08-08 20:56:50 on dist-test-slave-c12x"
server_key: "afdae8f529bc8c8117ffe63464fd8eb8"
server_key_iv: "2f89cde2fceb525eca53ec7226c38f2b"
server_key_version: "encryptionkey@0"
I20260808 20:56:50.504966 27971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "64d06fbc23484fab9905b6e5172142de"
format_stamp: "Formatted at 2026-08-08 20:56:50 on dist-test-slave-c12x"
server_key: "afdae8f529bc8c8117ffe63464fd8eb8"
server_key_iv: "2f89cde2fceb525eca53ec7226c38f2b"
server_key_version: "encryptionkey@0"
I20260808 20:56:50.506367 27971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:50.507117 27992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:50.507287 27971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:50.507395 27971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "64d06fbc23484fab9905b6e5172142de"
format_stamp: "Formatted at 2026-08-08 20:56:50 on dist-test-slave-c12x"
server_key: "afdae8f529bc8c8117ffe63464fd8eb8"
server_key_iv: "2f89cde2fceb525eca53ec7226c38f2b"
server_key_version: "encryptionkey@0"
I20260808 20:56:50.507507 27971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:50.549851 27971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:50.552407 27971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:50.552657 27971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:50.553341 27971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:50.553440 27971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:50.553495 27971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:50.553531 27971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222610, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:50.562625 27971 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:44185
I20260808 20:56:50.562711 28165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:44185 every 8 connection(s)
I20260808 20:56:50.563125 27971 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:50.564210 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.553903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:47043 (local address 127.15.232.190:43705)
0808 20:56:50.553980 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.553982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.554205 (+ 223us) server_negotiation.cc:407] Connection header received
0808 20:56:50.554769 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.554770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.554783 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.554837 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:50.555778 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.556298 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.557526 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.557783 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.557854 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.560880 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:50.560881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:50.560881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:50.560889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:50.562514 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.563037 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.563039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.563039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.563115 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.563541 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.563543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.563543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.563703 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:50.563765 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.564066 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.564145 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 20:56:50.564401 28007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.553846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:43705 (local address 127.15.232.131:47043)
0808 20:56:50.554055 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.554109 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.554647 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.554940 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.554950 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.555409 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:50.555675 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.555680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.556416 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.556417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.557406 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.557409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.557928 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.557929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.558034 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.558607 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:50.558616 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:50.560772 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:50.562608 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.562614 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.562616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.562943 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.563266 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.563267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.563267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.563426 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.563803 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:50.563809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:50.563937 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.564028 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.564161 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":105,"threads_started":1}
I20260808 20:56:50.564884 28168 heartbeater.cc:347] Connected to a master server at 127.15.232.190:43705
I20260808 20:56:50.564973 28168 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:50.565162 28168 heartbeater.cc:511] Master 127.15.232.190:43705 requested a full tablet report, sending...
I20260808 20:56:50.565550 27462 ts_manager.cc:194] Registered new tserver with Master: 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185)
I20260808 20:56:50.565936 27462 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:47043
I20260808 20:56:50.570747 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 27971
I20260808 20:56:50.570861 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:50.571103 16290 external_mini_cluster.cc:1428] Setting key 85f0c2df0396a6ab3dd5cc1e4ed7a492
I20260808 20:56:50.571305 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.566054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:36799 (local address 127.15.232.190:43705)
0808 20:56:50.566091 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.566093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.566190 (+ 97us) server_negotiation.cc:407] Connection header received
0808 20:56:50.566239 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.566240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.566251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.566305 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:50.566656 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.567122 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.567846 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.568039 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.568093 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.568908 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:50.568909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:50.568909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:50.568916 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:50.570114 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.570419 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.570419 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.570419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.570469 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.570687 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.570688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.570688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.570836 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:50.570911 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.571118 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.571200 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 20:56:50.571318 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.565995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:43705 (local address 127.15.232.131:36799)
0808 20:56:50.566127 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.566137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.566200 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.566316 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.566317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.566523 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:50.566582 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.566584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.567209 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.567210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.567774 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.567776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.568112 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.568112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.568163 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.568554 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:50.568558 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:50.568834 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:50.570184 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.570185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.570185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.570354 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.570568 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:50.570568 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:50.570569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:50.570621 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:50.570961 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:50.570962 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:50.571005 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.571067 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.571181 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260808 20:56:50.572223 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 20:56:50.595347 27563 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:50.589603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.15.232.190:43705)
0808 20:56:50.589674 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.589676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.589686 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:50.589750 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.589751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.589761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.589812 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:50.590366 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.590876 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.591746 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.591929 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.591985 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.595214 (+ 3229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53412: BlockingRecv error: recv got EOF from 127.0.0.1:53412 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 20:56:50 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:50.603196 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.596165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.15.232.190:43705)
0808 20:56:50.596230 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.596232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.596328 (+ 96us) server_negotiation.cc:407] Connection header received
0808 20:56:50.596398 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.596399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.596408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.596465 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:50.596951 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.597437 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.598150 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.598356 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.598435 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.600643 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:50.600644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:50.600645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:50.600653 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:50.602037 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.602383 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.602384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.602385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.602447 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:50.602663 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:50.602664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:50.602665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:50.602789 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:50.602903 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.602997 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.603087 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 20:56:50.605046 27462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:50.606765 27462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:50.613994 28197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.609574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:44185 (local address 127.0.0.1:34896)
0808 20:56:50.609910 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.609918 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.609954 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.610192 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.610196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.610198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:50.610276 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.610278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.611483 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.611484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.612746 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.612749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.613640 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.613641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.613736 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.613752 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.613834 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.613897 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":69,"threads_started":1}
I20260808 20:56:50.614024 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.609785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34896 (local address 127.15.232.131:44185)
0808 20:56:50.609951 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.609957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.609968 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:56:50.610023 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.610026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.610043 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.610154 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:50.610450 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.611353 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.612835 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.613536 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.613647 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.613721 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.613863 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.613913 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260808 20:56:50.614167 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.609378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45279 (local address 127.0.0.1:46252)
0808 20:56:50.609549 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.609617 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.609689 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.610466 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.610467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.610468 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:50.610524 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.610526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.611955 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.611956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.612962 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.612964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.613909 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.613910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.613981 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.613986 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.614018 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.614084 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260808 20:56:50.614562 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.609525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.15.232.130:45279)
0808 20:56:50.610319 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.610323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.610333 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:50.610371 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.610373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.610389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.610467 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:50.610616 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.611864 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.613045 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.613776 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.614302 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.614371 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.614397 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.614449 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":89,"threads_started":1}
I20260808 20:56:50.615144 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.609378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:37431 (local address 127.0.0.1:34224)
0808 20:56:50.609790 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.609799 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.609840 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.610590 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.610590 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.610593 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:50.610631 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.610632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.613039 (+ 2407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.613039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.613795 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.613797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.614892 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.614893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.614965 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.614970 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.615032 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.615078 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":4,"thread_start_us":54,"threads_started":1}
I20260808 20:56:50.615234 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.609520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.15.232.129:37431)
0808 20:56:50.610089 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.610093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.610103 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:50.610140 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.610143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.610157 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.610580 (+ 423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:50.610721 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.611859 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.613889 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.614753 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.614882 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.614938 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.615062 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.615104 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":94,"threads_started":1}
I20260808 20:56:50.615545 27872 tablet_service.cc:1467] Processing CreateTablet for tablet 7d56e26228bd4e0e99abc81b791ed5a4 (DEFAULT_TABLE table=test-table [id=69c1cf4cc46c421e93509788cf3021f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:50.615948 27872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d56e26228bd4e0e99abc81b791ed5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:50.616057 28073 tablet_service.cc:1467] Processing CreateTablet for tablet 7d56e26228bd4e0e99abc81b791ed5a4 (DEFAULT_TABLE table=test-table [id=69c1cf4cc46c421e93509788cf3021f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:50.616108 27674 tablet_service.cc:1467] Processing CreateTablet for tablet 7d56e26228bd4e0e99abc81b791ed5a4 (DEFAULT_TABLE table=test-table [id=69c1cf4cc46c421e93509788cf3021f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:50.616425 27674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d56e26228bd4e0e99abc81b791ed5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:50.616425 28073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d56e26228bd4e0e99abc81b791ed5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:50.619499 28204 tablet_bootstrap.cc:492] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: Bootstrap starting.
I20260808 20:56:50.619897 28205 tablet_bootstrap.cc:492] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: Bootstrap starting.
I20260808 20:56:50.620249 28204 tablet_bootstrap.cc:654] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:50.620633 28204 log.cc:824] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:50.620529 28206 tablet_bootstrap.cc:492] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: Bootstrap starting.
I20260808 20:56:50.620843 28205 tablet_bootstrap.cc:654] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:50.621181 28205 log.cc:824] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:50.621299 28206 tablet_bootstrap.cc:654] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:50.621637 28206 log.cc:824] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:50.621974 28205 tablet_bootstrap.cc:492] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: No bootstrap required, opened a new log
I20260808 20:56:50.622059 28205 ts_tablet_manager.cc:1397] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:56:50.622448 28204 tablet_bootstrap.cc:492] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: No bootstrap required, opened a new log
I20260808 20:56:50.622475 28206 tablet_bootstrap.cc:492] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: No bootstrap required, opened a new log
I20260808 20:56:50.622527 28204 ts_tablet_manager.cc:1397] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 20:56:50.622555 28206 ts_tablet_manager.cc:1397] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:50.624046 28206 raft_consensus.cc:348] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.624209 28206 raft_consensus.cc:374] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:50.624257 28206 raft_consensus.cc:729] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c5b28b4acc4cb58e89e035db0cacfe, State: Initialized, Role: FOLLOWER
I20260808 20:56:50.624173 28205 raft_consensus.cc:348] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.624344 28205 raft_consensus.cc:374] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:50.624396 28205 raft_consensus.cc:729] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d06fbc23484fab9905b6e5172142de, State: Initialized, Role: FOLLOWER
I20260808 20:56:50.624368 28206 consensus_queue.cc:260] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.624545 28205 consensus_queue.cc:260] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.624670 28206 ts_tablet_manager.cc:1428] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 20:56:50.624596 28204 raft_consensus.cc:348] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.624758 28204 raft_consensus.cc:374] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:50.624802 28204 raft_consensus.cc:729] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4704fd423bb4ae4a2d7c7e9ee41002c, State: Initialized, Role: FOLLOWER
I20260808 20:56:50.624817 28205 ts_tablet_manager.cc:1428] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 20:56:50.624840 27768 heartbeater.cc:503] Master 127.15.232.190:43705 was elected leader, sending a full tablet report...
I20260808 20:56:50.624948 28168 heartbeater.cc:503] Master 127.15.232.190:43705 was elected leader, sending a full tablet report...
I20260808 20:56:50.624912 28204 consensus_queue.cc:260] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.625209 28204 ts_tablet_manager.cc:1428] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 20:56:50.625218 27959 heartbeater.cc:503] Master 127.15.232.190:43705 was elected leader, sending a full tablet report...
W20260808 20:56:50.693007 27769 tablet.cc:2369] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 20:56:50.813786 28169 tablet.cc:2369] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:50.826184 28220 raft_consensus.cc:482] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:50.826342 28220 raft_consensus.cc:504] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.826726 28220 leader_election.cc:288] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185)
I20260808 20:56:50.831048 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.827035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:44049 (local address 127.15.232.129:37431)
0808 20:56:50.827087 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.827089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.827239 (+ 150us) server_negotiation.cc:407] Connection header received
0808 20:56:50.827269 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.827270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.827279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.827350 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:50.827510 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.828303 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.829940 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.830492 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.830577 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.830586 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.830919 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.830974 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 20:56:50.831048 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.827066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:43611 (local address 127.15.232.131:44185)
0808 20:56:50.827133 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.827135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.827153 (+ 18us) server_negotiation.cc:407] Connection header received
0808 20:56:50.827186 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.827186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.827197 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.827288 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:50.827544 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.828280 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.829940 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.830511 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.830577 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.830586 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.830919 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.830975 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 20:56:50.831030 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.826964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:44185 (local address 127.15.232.130:43611)
0808 20:56:50.827027 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.827048 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.827098 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.827309 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.827310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.827311 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:50.827407 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.827410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.828392 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.828394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.829829 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.829833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.830595 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.830596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.830783 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.830786 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.830851 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.830918 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":108}
I20260808 20:56:50.831039 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.826974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:37431 (local address 127.15.232.130:44049)
0808 20:56:50.827148 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:50.827158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:50.827215 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:50.827359 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:50.827359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:50.827361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:50.827425 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.827426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.828393 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.828394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.829829 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:50.829833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.830596 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.830596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:50.830770 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.830778 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 20:56:50.830851 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 20:56:50.830918 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":75,"thread_start_us":97,"threads_started":1}
I20260808 20:56:50.831521 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d56e26228bd4e0e99abc81b791ed5a4" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d06fbc23484fab9905b6e5172142de" is_pre_election: true
I20260808 20:56:50.831735 28100 raft_consensus.cc:2454] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 0.
I20260808 20:56:50.831712 27699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d56e26228bd4e0e99abc81b791ed5a4" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" is_pre_election: true
I20260808 20:56:50.831840 27699 raft_consensus.cc:2454] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 0.
I20260808 20:56:50.832017 27799 leader_election.cc:302] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d06fbc23484fab9905b6e5172142de, e4704fd423bb4ae4a2d7c7e9ee41002c; no voters:
I20260808 20:56:50.832180 28220 raft_consensus.cc:2781] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:50.832261 28220 raft_consensus.cc:482] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:50.832281 28220 raft_consensus.cc:3037] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:50.833426 28220 raft_consensus.cc:504] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.833606 28220 leader_election.cc:288] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 election: Requested vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185)
I20260808 20:56:50.833750 27699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d56e26228bd4e0e99abc81b791ed5a4" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe"
I20260808 20:56:50.833822 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d56e26228bd4e0e99abc81b791ed5a4" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d06fbc23484fab9905b6e5172142de"
I20260808 20:56:50.833863 27699 raft_consensus.cc:3037] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:50.833894 28100 raft_consensus.cc:3037] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:50.834743 27699 raft_consensus.cc:2454] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 1.
I20260808 20:56:50.834749 28100 raft_consensus.cc:2454] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 1.
I20260808 20:56:50.835011 27799 leader_election.cc:302] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d06fbc23484fab9905b6e5172142de, e4704fd423bb4ae4a2d7c7e9ee41002c; no voters:
I20260808 20:56:50.835135 28220 raft_consensus.cc:2781] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:50.835304 28220 raft_consensus.cc:686] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 LEADER]: Becoming Leader. State: Replica: e4704fd423bb4ae4a2d7c7e9ee41002c, State: Running, Role: LEADER
I20260808 20:56:50.835425 28220 consensus_queue.cc:237] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.836557 27461 catalog_manager.cc:5495] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c reported cstate change: term changed from 0 to 1, leader changed from <none> to e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130). New cstate: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } health_report { overall_health: UNKNOWN } } }
W20260808 20:56:50.870924 27960 tablet.cc:2369] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:56:50.898846 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:50.896243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.15.232.130:45279)
0808 20:56:50.896291 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:50.896298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:50.896331 (+ 33us) server_negotiation.cc:407] Connection header received
0808 20:56:50.896402 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:50.896403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:50.896419 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:50.896503 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:50.896605 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.897117 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.897937 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:50.898162 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:50.898227 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:50.898513 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:50.898626 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:50.898704 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:50.898761 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 20:56:50.898783 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 20:56:50.900983 27699 raft_consensus.cc:1261] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Refusing update from remote peer e4704fd423bb4ae4a2d7c7e9ee41002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:50.901026 28100 raft_consensus.cc:1261] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Refusing update from remote peer e4704fd423bb4ae4a2d7c7e9ee41002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:50.901394 28220 consensus_queue.cc:1035] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:50.901576 28224 consensus_queue.cc:1035] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:50.906786 28230 mvcc.cc:205] Tried to move back new op lower bound from 7316367814247448576 to 7316367813982900224. Current Snapshot: MvccSnapshot[applied={T|T < 7316367814247448576 or (T in {7316367814247448576})}]
I20260808 20:56:50.907054 27462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:50.907150 27462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:50.908039 27462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a: Sending DeleteTablet for 3 replicas of tablet 7d56e26228bd4e0e99abc81b791ed5a4
I20260808 20:56:50.908399 27674 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d56e26228bd4e0e99abc81b791ed5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:50 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34224
I20260808 20:56:50.908443 27872 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d56e26228bd4e0e99abc81b791ed5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:50 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:46252
I20260808 20:56:50.908694 28240 tablet_replica.cc:330] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: stopping tablet replica
I20260808 20:56:50.908706 28073 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d56e26228bd4e0e99abc81b791ed5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:50 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34896
I20260808 20:56:50.908794 28240 raft_consensus.cc:2229] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:50.908903 28240 raft_consensus.cc:2258] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:50.909219 28241 tablet_replica.cc:330] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: stopping tablet replica
I20260808 20:56:50.909171 27462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:50.909327 27462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:50.909461 28240 ts_tablet_manager.cc:1905] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:50.909650 28241 raft_consensus.cc:2229] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:50.909955 28241 raft_consensus.cc:2258] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:50.910554 28241 ts_tablet_manager.cc:1905] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:50.910915 27872 tablet_service.cc:1467] Processing CreateTablet for tablet 96c899aa063a4bd79dd23367d5ebe833 (DEFAULT_TABLE table=test-table [id=c2482620ed704743a8327754c3945c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:50.911049 27872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96c899aa063a4bd79dd23367d5ebe833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:50.911496 28240 ts_tablet_manager.cc:1918] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:50.911580 28240 log.cc:1196] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/7d56e26228bd4e0e99abc81b791ed5a4
I20260808 20:56:50.911856 28240 ts_tablet_manager.cc:1939] T 7d56e26228bd4e0e99abc81b791ed5a4 P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting consensus metadata
I20260808 20:56:50.912302 27444 catalog_manager.cc:4841] TS e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130:45279): tablet 7d56e26228bd4e0e99abc81b791ed5a4 (table test-table [id=69c1cf4cc46c421e93509788cf3021f8]) successfully deleted
I20260808 20:56:50.912357 28204 tablet_bootstrap.cc:492] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: Bootstrap starting.
I20260808 20:56:50.912786 28204 tablet_bootstrap.cc:654] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:50.912861 28241 ts_tablet_manager.cc:1918] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:50.912940 28241 log.cc:1196] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/7d56e26228bd4e0e99abc81b791ed5a4
I20260808 20:56:50.913175 27674 tablet_service.cc:1467] Processing CreateTablet for tablet 96c899aa063a4bd79dd23367d5ebe833 (DEFAULT_TABLE table=test-table [id=c2482620ed704743a8327754c3945c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:50.913265 28241 ts_tablet_manager.cc:1939] T 7d56e26228bd4e0e99abc81b791ed5a4 P 64d06fbc23484fab9905b6e5172142de: Deleting consensus metadata
I20260808 20:56:50.913316 27674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96c899aa063a4bd79dd23367d5ebe833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:50.914896 28206 tablet_bootstrap.cc:492] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: Bootstrap starting.
I20260808 20:56:50.914999 27444 catalog_manager.cc:4841] TS 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185): tablet 7d56e26228bd4e0e99abc81b791ed5a4 (table test-table [id=69c1cf4cc46c421e93509788cf3021f8]) successfully deleted
I20260808 20:56:50.915140 28073 tablet_service.cc:1467] Processing CreateTablet for tablet 96c899aa063a4bd79dd23367d5ebe833 (DEFAULT_TABLE table=test-table [id=c2482620ed704743a8327754c3945c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:50.915252 28073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96c899aa063a4bd79dd23367d5ebe833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:50.915354 28206 tablet_bootstrap.cc:654] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:50.915992 28204 tablet_bootstrap.cc:492] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: No bootstrap required, opened a new log
I20260808 20:56:50.916047 28204 ts_tablet_manager.cc:1397] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260808 20:56:50.916234 28204 raft_consensus.cc:348] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.916330 28204 raft_consensus.cc:374] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:50.916357 28204 raft_consensus.cc:729] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4704fd423bb4ae4a2d7c7e9ee41002c, State: Initialized, Role: FOLLOWER
I20260808 20:56:50.916407 28204 consensus_queue.cc:260] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.916447 28206 tablet_bootstrap.cc:492] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: No bootstrap required, opened a new log
I20260808 20:56:50.916487 28206 ts_tablet_manager.cc:1397] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 20:56:50.916512 28204 ts_tablet_manager.cc:1428] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:50.916555 28205 tablet_bootstrap.cc:492] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: Bootstrap starting.
I20260808 20:56:50.916641 28206 raft_consensus.cc:348] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.916714 28239 tablet_replica.cc:330] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: stopping tablet replica
I20260808 20:56:50.916733 28206 raft_consensus.cc:374] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:50.916755 28206 raft_consensus.cc:729] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c5b28b4acc4cb58e89e035db0cacfe, State: Initialized, Role: FOLLOWER
I20260808 20:56:50.916796 28206 consensus_queue.cc:260] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.916828 28239 raft_consensus.cc:2229] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:50.916893 28206 ts_tablet_manager.cc:1428] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:50.916953 28205 tablet_bootstrap.cc:654] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:50.916953 28239 raft_consensus.cc:2258] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:50.917515 28239 ts_tablet_manager.cc:1905] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:50.918769 28205 tablet_bootstrap.cc:492] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: No bootstrap required, opened a new log
I20260808 20:56:50.918831 28205 ts_tablet_manager.cc:1397] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:50.919003 28205 raft_consensus.cc:348] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.919077 28205 raft_consensus.cc:374] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:50.919095 28205 raft_consensus.cc:729] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d06fbc23484fab9905b6e5172142de, State: Initialized, Role: FOLLOWER
I20260808 20:56:50.919137 28205 consensus_queue.cc:260] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:50.919160 28239 ts_tablet_manager.cc:1918] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:50.919234 28239 log.cc:1196] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/7d56e26228bd4e0e99abc81b791ed5a4
I20260808 20:56:50.919245 28205 ts_tablet_manager.cc:1428] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 20:56:50.919970 28239 ts_tablet_manager.cc:1939] T 7d56e26228bd4e0e99abc81b791ed5a4 P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting consensus metadata
I20260808 20:56:50.920416 27445 catalog_manager.cc:4841] TS 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431): tablet 7d56e26228bd4e0e99abc81b791ed5a4 (table test-table [id=69c1cf4cc46c421e93509788cf3021f8]) successfully deleted
I20260808 20:56:51.081910 28219 raft_consensus.cc:482] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:51.082038 28219 raft_consensus.cc:504] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.082553 28219 leader_election.cc:288] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130:45279)
I20260808 20:56:51.086699 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.082878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:53563 (local address 127.15.232.130:45279)
0808 20:56:51.082924 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.082926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.083023 (+ 97us) server_negotiation.cc:407] Connection header received
0808 20:56:51.083097 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.083098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.083110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.083182 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:51.083344 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.084133 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.085767 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.086312 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.086384 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.086392 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.086583 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.086630 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 20:56:51.086709 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.082845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:52881 (local address 127.15.232.129:37431)
0808 20:56:51.082911 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.082912 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.082950 (+ 38us) server_negotiation.cc:407] Connection header received
0808 20:56:51.083053 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.083054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.083064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.083137 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:56:51.083349 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.084120 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.085768 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.086305 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.086384 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.086392 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.086610 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.086649 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 20:56:51.086740 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.082779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45279 (local address 127.15.232.131:53563)
0808 20:56:51.082934 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:51.082942 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:51.082996 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:51.083181 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:51.083182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:51.083184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:51.083262 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.083264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.084262 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.084264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.085664 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.085667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.086406 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.086408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.086507 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.086510 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:56:51.086565 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 20:56:51.086602 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":81,"thread_start_us":89,"threads_started":1}
I20260808 20:56:51.086750 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.082741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:37431 (local address 127.15.232.131:52881)
0808 20:56:51.082883 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:51.082895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:51.082970 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:51.083174 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:51.083176 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:51.083179 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:56:51.083261 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.083266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.084262 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.084264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.085664 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.085667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.086408 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.086409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.086485 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.086490 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:56:51.086556 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 20:56:51.086602 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":79,"thread_start_us":64,"threads_started":1}
I20260808 20:56:51.087059 27901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c899aa063a4bd79dd23367d5ebe833" candidate_uuid: "64d06fbc23484fab9905b6e5172142de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" is_pre_election: true
I20260808 20:56:51.087245 27901 raft_consensus.cc:2454] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d06fbc23484fab9905b6e5172142de in term 0.
I20260808 20:56:51.087232 27699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c899aa063a4bd79dd23367d5ebe833" candidate_uuid: "64d06fbc23484fab9905b6e5172142de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" is_pre_election: true
I20260808 20:56:51.087343 27699 raft_consensus.cc:2454] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d06fbc23484fab9905b6e5172142de in term 0.
I20260808 20:56:51.087486 27995 leader_election.cc:302] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d06fbc23484fab9905b6e5172142de, e4704fd423bb4ae4a2d7c7e9ee41002c; no voters:
I20260808 20:56:51.087649 28219 raft_consensus.cc:2781] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:51.087702 28219 raft_consensus.cc:482] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:51.087718 28219 raft_consensus.cc:3037] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.088375 28219 raft_consensus.cc:504] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.088526 28219 leader_election.cc:288] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [CANDIDATE]: Term 1 election: Requested vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130:45279)
I20260808 20:56:51.088716 27699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c899aa063a4bd79dd23367d5ebe833" candidate_uuid: "64d06fbc23484fab9905b6e5172142de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe"
I20260808 20:56:51.088800 27699 raft_consensus.cc:3037] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.088737 27901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96c899aa063a4bd79dd23367d5ebe833" candidate_uuid: "64d06fbc23484fab9905b6e5172142de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c"
I20260808 20:56:51.088835 27901 raft_consensus.cc:3037] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.089485 27699 raft_consensus.cc:2454] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d06fbc23484fab9905b6e5172142de in term 1.
I20260808 20:56:51.089558 27901 raft_consensus.cc:2454] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d06fbc23484fab9905b6e5172142de in term 1.
I20260808 20:56:51.089679 27998 leader_election.cc:302] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48c5b28b4acc4cb58e89e035db0cacfe, 64d06fbc23484fab9905b6e5172142de; no voters:
I20260808 20:56:51.089781 28219 raft_consensus.cc:2781] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:51.089912 28219 raft_consensus.cc:686] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 1 LEADER]: Becoming Leader. State: Replica: 64d06fbc23484fab9905b6e5172142de, State: Running, Role: LEADER
I20260808 20:56:51.089988 28219 consensus_queue.cc:237] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.090744 27461 catalog_manager.cc:5495] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de reported cstate change: term changed from 0 to 1, leader changed from <none> to 64d06fbc23484fab9905b6e5172142de (127.15.232.131). New cstate: current_term: 1 leader_uuid: "64d06fbc23484fab9905b6e5172142de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } health_report { overall_health: HEALTHY } } }
Aug 08 20:56:51 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222609, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.190@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:51.139546 28257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.132173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:43705 (local address 127.0.0.1:53422)
0808 20:56:51.132311 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:51.132322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:51.132404 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:51.132577 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:51.132578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:51.132882 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:51.132929 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.132931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.133624 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.133625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.134197 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.134199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.134571 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.134572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.134630 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.135211 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:51.135215 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:51.136891 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:51.138445 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:51.138452 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:51.138454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:51.138665 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:51.138912 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:51.138912 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:51.138913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:51.138969 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:51.139204 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:51.139211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:51.139255 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 20:56:51.139314 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 20:56:51.139410 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260808 20:56:51.139639 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.132251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53422 (local address 127.15.232.190:43705)
0808 20:56:51.132418 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.132420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.132446 (+ 26us) server_negotiation.cc:407] Connection header received
0808 20:56:51.132471 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.132472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.132482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.132538 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:51.133009 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.133547 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.134276 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.134496 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.134554 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.137016 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:51.137018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:51.137018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:51.137026 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:51.138370 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:51.138767 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:51.138773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:51.138774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:51.138839 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:51.139027 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:51.139028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:51.139028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:51.139139 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:51.139194 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.139351 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.139539 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260808 20:56:51.141723 27461 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:53422:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 20:56:51.141957 27461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:51.143563 27674 tablet_service.cc:1467] Processing CreateTablet for tablet a5578d644c24497a8c2aa3e0d6948b93 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a769e156f91a474b8979b857948068db]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 20:56:51.143627 28073 tablet_service.cc:1467] Processing CreateTablet for tablet a5578d644c24497a8c2aa3e0d6948b93 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a769e156f91a474b8979b857948068db]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 20:56:51.143623 27872 tablet_service.cc:1467] Processing CreateTablet for tablet a5578d644c24497a8c2aa3e0d6948b93 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a769e156f91a474b8979b857948068db]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 20:56:51.143752 28073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5578d644c24497a8c2aa3e0d6948b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:51.143752 27674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5578d644c24497a8c2aa3e0d6948b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:51.143805 27872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5578d644c24497a8c2aa3e0d6948b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:51.145042 28204 tablet_bootstrap.cc:492] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: Bootstrap starting.
I20260808 20:56:51.145293 28205 tablet_bootstrap.cc:492] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: Bootstrap starting.
I20260808 20:56:51.145474 28204 tablet_bootstrap.cc:654] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:51.145740 28205 tablet_bootstrap.cc:654] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:51.146090 28204 tablet_bootstrap.cc:492] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: No bootstrap required, opened a new log
I20260808 20:56:51.146132 28206 tablet_bootstrap.cc:492] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: Bootstrap starting.
I20260808 20:56:51.146162 28204 ts_tablet_manager.cc:1397] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:51.146297 28204 raft_consensus.cc:348] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.146384 28204 raft_consensus.cc:374] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:51.146409 28204 raft_consensus.cc:729] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4704fd423bb4ae4a2d7c7e9ee41002c, State: Initialized, Role: FOLLOWER
I20260808 20:56:51.146430 28205 tablet_bootstrap.cc:492] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: No bootstrap required, opened a new log
I20260808 20:56:51.146472 28205 ts_tablet_manager.cc:1397] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:51.146459 28204 consensus_queue.cc:260] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.146556 28206 tablet_bootstrap.cc:654] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:51.146605 28204 ts_tablet_manager.cc:1428] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:51.146622 28205 raft_consensus.cc:348] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.146689 28205 raft_consensus.cc:374] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:51.146708 28205 raft_consensus.cc:729] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d06fbc23484fab9905b6e5172142de, State: Initialized, Role: FOLLOWER
I20260808 20:56:51.146651 28220 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.146761 28220 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:51.146781 28205 consensus_queue.cc:260] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.146870 28205 ts_tablet_manager.cc:1428] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:51.146915 28219 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.146998 28219 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:51.147408 28206 tablet_bootstrap.cc:492] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: No bootstrap required, opened a new log
I20260808 20:56:51.147490 28206 ts_tablet_manager.cc:1397] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:51.147641 28206 raft_consensus.cc:348] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.147701 28206 raft_consensus.cc:374] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:51.147715 28206 raft_consensus.cc:729] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c5b28b4acc4cb58e89e035db0cacfe, State: Initialized, Role: FOLLOWER
I20260808 20:56:51.147773 28206 consensus_queue.cc:260] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.147869 28206 ts_tablet_manager.cc:1428] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:51.147943 28217 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.148023 28217 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:51.160791 28220 raft_consensus.cc:482] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:51.160946 28220 raft_consensus.cc:504] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.161166 28220 leader_election.cc:288] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185)
I20260808 20:56:51.161397 27699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5578d644c24497a8c2aa3e0d6948b93" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" is_pre_election: true
I20260808 20:56:51.161397 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5578d644c24497a8c2aa3e0d6948b93" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d06fbc23484fab9905b6e5172142de" is_pre_election: true
I20260808 20:56:51.161506 27699 raft_consensus.cc:2454] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 0.
I20260808 20:56:51.161504 28100 raft_consensus.cc:2454] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 0.
I20260808 20:56:51.161664 27802 leader_election.cc:302] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48c5b28b4acc4cb58e89e035db0cacfe, e4704fd423bb4ae4a2d7c7e9ee41002c; no voters:
I20260808 20:56:51.161762 28220 raft_consensus.cc:2781] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:51.161803 28220 raft_consensus.cc:482] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:51.161828 28220 raft_consensus.cc:3037] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.162565 28220 raft_consensus.cc:504] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.162715 28220 leader_election.cc:288] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 election: Requested vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185)
I20260808 20:56:51.162911 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5578d644c24497a8c2aa3e0d6948b93" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d06fbc23484fab9905b6e5172142de"
I20260808 20:56:51.162911 27699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5578d644c24497a8c2aa3e0d6948b93" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe"
I20260808 20:56:51.163004 27699 raft_consensus.cc:3037] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.163004 28100 raft_consensus.cc:3037] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.163901 27699 raft_consensus.cc:2454] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 1.
I20260808 20:56:51.163901 28100 raft_consensus.cc:2454] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 1.
I20260808 20:56:51.164119 27802 leader_election.cc:302] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48c5b28b4acc4cb58e89e035db0cacfe, e4704fd423bb4ae4a2d7c7e9ee41002c; no voters:
I20260808 20:56:51.164240 28220 raft_consensus.cc:2781] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:51.164315 28220 raft_consensus.cc:686] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 LEADER]: Becoming Leader. State: Replica: e4704fd423bb4ae4a2d7c7e9ee41002c, State: Running, Role: LEADER
I20260808 20:56:51.164373 28220 consensus_queue.cc:237] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.164696 28224 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: TxnStatusTablet state changed. Reason: New leader e4704fd423bb4ae4a2d7c7e9ee41002c. Latest consensus state: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.164789 28224 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: This TxnStatusTablet replica's current role is: LEADER
I20260808 20:56:51.165009 28266 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 20:56:51.165150 28266 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 20:56:51.165139 27462 catalog_manager.cc:5495] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c reported cstate change: term changed from 0 to 1, leader changed from <none> to e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130). New cstate: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } health_report { overall_health: UNKNOWN } } }
I20260808 20:56:51.173272 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.170324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46278 (local address 127.15.232.130:45279)
0808 20:56:51.170390 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.170392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.170457 (+ 65us) server_negotiation.cc:407] Connection header received
0808 20:56:51.170544 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.170544 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.170555 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.170602 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:51.170725 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.171204 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.172312 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.172583 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.172680 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.172976 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.173017 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.173087 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.173167 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.173190 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 20:56:51.173290 28257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.170253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45279 (local address 127.0.0.1:46278)
0808 20:56:51.170403 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:51.170417 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:51.170479 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:51.170622 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:51.170623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:51.170624 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:51.170674 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.170677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.171286 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.171287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.172223 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.172226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.172744 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.172745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.172823 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.172842 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.173072 (+ 230us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.173109 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 20:56:51.173164 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 20:56:51.173217 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260808 20:56:51.174615 27699 raft_consensus.cc:1261] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Refusing update from remote peer e4704fd423bb4ae4a2d7c7e9ee41002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:51.174849 28224 consensus_queue.cc:1035] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:51.174907 28100 raft_consensus.cc:1261] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Refusing update from remote peer e4704fd423bb4ae4a2d7c7e9ee41002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:51.175107 28224 consensus_queue.cc:1035] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:51.175539 28217 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: TxnStatusTablet state changed. Reason: New leader e4704fd423bb4ae4a2d7c7e9ee41002c. Latest consensus state: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.175647 28217 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:51.175894 28219 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: TxnStatusTablet state changed. Reason: New leader e4704fd423bb4ae4a2d7c7e9ee41002c. Latest consensus state: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.176017 28219 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:51.176343 28224 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.176420 28224 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: This TxnStatusTablet replica's current role is: LEADER
I20260808 20:56:51.176779 28220 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.176857 28220 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P e4704fd423bb4ae4a2d7c7e9ee41002c: This TxnStatusTablet replica's current role is: LEADER
I20260808 20:56:51.176895 28229 mvcc.cc:205] Tried to move back new op lower bound from 7316367815369457664 to 7316367815329652736. Current Snapshot: MvccSnapshot[applied={T|T < 7316367815369457664}]
I20260808 20:56:51.176992 28217 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.177062 28217 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P 48c5b28b4acc4cb58e89e035db0cacfe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:51.177134 28219 tablet_replica.cc:439] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } } }
I20260808 20:56:51.177224 28219 tablet_replica.cc:442] T a5578d644c24497a8c2aa3e0d6948b93 P 64d06fbc23484fab9905b6e5172142de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 20:56:51.180877 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.178457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34902 (local address 127.15.232.131:44185)
0808 20:56:51.178507 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.178509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.178519 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:51.178567 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.178568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.178587 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.178669 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:51.178796 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.179267 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.180070 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.180284 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.180371 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.180544 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.180653 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.180738 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.180778 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.180800 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 20:56:51.186118 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.183495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:47821 (local address 127.15.232.130:45279)
0808 20:56:51.183576 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.183577 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.183676 (+ 99us) server_negotiation.cc:407] Connection header received
0808 20:56:51.183734 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.183735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.183746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.183794 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:51.183937 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.184367 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.185459 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.185657 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.185712 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.185908 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.185932 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.185970 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.186047 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.186064 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 20:56:51.186179 28274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.183403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45279 (local address 127.15.232.131:47821)
0808 20:56:51.183608 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:51.183618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:51.183678 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:51.183817 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:51.183818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:51.183819 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:51.183878 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.183879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.184432 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.184433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.185373 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.185376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.185749 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.185750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.185813 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.185828 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.185990 (+ 162us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.185992 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 20:56:51.186046 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 20:56:51.186090 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":109,"threads_started":1}
I20260808 20:56:51.188864 27901 raft_consensus.cc:1261] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Refusing update from remote peer 64d06fbc23484fab9905b6e5172142de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:51.189083 27699 raft_consensus.cc:1261] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Refusing update from remote peer 64d06fbc23484fab9905b6e5172142de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:51.189379 28248 consensus_queue.cc:1035] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:51.189548 28219 consensus_queue.cc:1035] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [LEADER]: Connected to new peer: Peer: permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:51.192642 28227 mvcc.cc:205] Tried to move back new op lower bound from 7316367815427416064 to 7316367815025471488. Current Snapshot: MvccSnapshot[applied={T|T < 7316367815427416064}]
I20260808 20:56:51.199590 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.197034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:58461 (local address 127.15.232.131:44185)
0808 20:56:51.197078 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.197080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.197089 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:56:51.197112 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.197113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.197124 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.197175 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:51.197271 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.197999 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.198792 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.199051 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.199144 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.199319 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.199355 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.199406 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.199495 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.199519 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 20:56:51.199648 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.196826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:44185 (local address 127.15.232.130:58461)
0808 20:56:51.196997 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:51.197009 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:51.197066 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:51.197188 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:51.197189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:51.197190 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:51.197234 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.197235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.198099 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.198100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.198674 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.198677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.199148 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.199149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.199210 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.199229 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.199417 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.199419 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 20:56:51.199482 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 20:56:51.199561 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260808 20:56:51.208837 27462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:51.208964 27462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:51.209710 27462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a: Sending DeleteTablet for 3 replicas of tablet 96c899aa063a4bd79dd23367d5ebe833
I20260808 20:56:51.210008 27674 tablet_service.cc:1514] Processing DeleteTablet for tablet 96c899aa063a4bd79dd23367d5ebe833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:51 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34224
I20260808 20:56:51.210104 28239 tablet_replica.cc:330] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: stopping tablet replica
I20260808 20:56:51.210147 28239 raft_consensus.cc:2229] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:51.210182 28239 raft_consensus.cc:2258] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:51.210328 28073 tablet_service.cc:1514] Processing DeleteTablet for tablet 96c899aa063a4bd79dd23367d5ebe833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:51 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34896
I20260808 20:56:51.210448 28241 tablet_replica.cc:330] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: stopping tablet replica
I20260808 20:56:51.210443 27872 tablet_service.cc:1514] Processing DeleteTablet for tablet 96c899aa063a4bd79dd23367d5ebe833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:51 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:46252
I20260808 20:56:51.210510 28241 raft_consensus.cc:2229] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:51.210542 28239 ts_tablet_manager.cc:1905] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:51.210614 28241 raft_consensus.cc:2258] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:51.212220 28240 tablet_replica.cc:330] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: stopping tablet replica
I20260808 20:56:51.212306 28240 raft_consensus.cc:2229] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:51.212322 28239 ts_tablet_manager.cc:1918] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:51.212345 28240 raft_consensus.cc:2258] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:51.212375 28239 log.cc:1196] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/96c899aa063a4bd79dd23367d5ebe833
I20260808 20:56:51.212646 28239 ts_tablet_manager.cc:1939] T 96c899aa063a4bd79dd23367d5ebe833 P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting consensus metadata
I20260808 20:56:51.212769 28241 ts_tablet_manager.cc:1905] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:51.212987 27445 catalog_manager.cc:4841] TS 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431): tablet 96c899aa063a4bd79dd23367d5ebe833 (table test-table [id=c2482620ed704743a8327754c3945c11]) successfully deleted
I20260808 20:56:51.214159 28241 ts_tablet_manager.cc:1918] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:51.214216 28241 log.cc:1196] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/96c899aa063a4bd79dd23367d5ebe833
I20260808 20:56:51.214427 28241 ts_tablet_manager.cc:1939] T 96c899aa063a4bd79dd23367d5ebe833 P 64d06fbc23484fab9905b6e5172142de: Deleting consensus metadata
I20260808 20:56:51.214747 27444 catalog_manager.cc:4841] TS 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185): tablet 96c899aa063a4bd79dd23367d5ebe833 (table test-table [id=c2482620ed704743a8327754c3945c11]) successfully deleted
I20260808 20:56:51.222115 28240 ts_tablet_manager.cc:1905] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:51.223963 28240 ts_tablet_manager.cc:1918] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:51.224031 28240 log.cc:1196] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/96c899aa063a4bd79dd23367d5ebe833
I20260808 20:56:51.224258 28240 ts_tablet_manager.cc:1939] T 96c899aa063a4bd79dd23367d5ebe833 P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting consensus metadata
I20260808 20:56:51.224769 27444 catalog_manager.cc:4841] TS e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130:45279): tablet 96c899aa063a4bd79dd23367d5ebe833 (table test-table [id=c2482620ed704743a8327754c3945c11]) successfully deleted
Aug 08 20:56:51 dist-test-slave-c12x krb5kdc[27398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 20:56:51 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:51 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 20:56:51.247655 28258 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:51.240529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.15.232.190:43705)
0808 20:56:51.240584 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.240586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.240768 (+ 182us) server_negotiation.cc:407] Connection header received
0808 20:56:51.240829 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.240830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.240843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.240906 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:51.241664 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.242456 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.243606 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.243887 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.243968 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.247585 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53426: BlockingRecv error: recv got EOF from 127.0.0.1:53426 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 20:56:51 dist-test-slave-c12x krb5kdc[27398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM
I20260808 20:56:51.257649 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.248438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.15.232.190:43705)
0808 20:56:51.248480 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.248482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.248728 (+ 246us) server_negotiation.cc:407] Connection header received
0808 20:56:51.248782 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.248783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.248794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.248857 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:51.249477 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.250256 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.251245 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.251632 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.251705 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.254569 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:51.254571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:51.254571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:51.254580 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:51.256317 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:51.256743 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:51.256744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:51.256745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:51.256814 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:51.257046 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:51.257047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:51.257048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:51.257197 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:51.257306 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.257456 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.257544 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 20:56:51.259730 27462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:56:51.259917 27462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:51.261454 27872 tablet_service.cc:1467] Processing CreateTablet for tablet 420dd629a9c84b26ac6526d5347438cb (DEFAULT_TABLE table=test-table [id=82447e8b12864eeab73ba5bc7d78e07d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:51.261462 28073 tablet_service.cc:1467] Processing CreateTablet for tablet 420dd629a9c84b26ac6526d5347438cb (DEFAULT_TABLE table=test-table [id=82447e8b12864eeab73ba5bc7d78e07d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:51.261601 27872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420dd629a9c84b26ac6526d5347438cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:51.261613 28073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420dd629a9c84b26ac6526d5347438cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:51.262976 28204 tablet_bootstrap.cc:492] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: Bootstrap starting.
I20260808 20:56:51.262998 28205 tablet_bootstrap.cc:492] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: Bootstrap starting.
I20260808 20:56:51.263276 28204 tablet_bootstrap.cc:654] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:51.263504 28205 tablet_bootstrap.cc:654] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:51.263962 27674 tablet_service.cc:1467] Processing CreateTablet for tablet 420dd629a9c84b26ac6526d5347438cb (DEFAULT_TABLE table=test-table [id=82447e8b12864eeab73ba5bc7d78e07d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:51.264094 27674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420dd629a9c84b26ac6526d5347438cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:51.264446 28204 tablet_bootstrap.cc:492] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: No bootstrap required, opened a new log
I20260808 20:56:51.264487 28204 ts_tablet_manager.cc:1397] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:56:51.264592 28204 raft_consensus.cc:348] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.264657 28204 raft_consensus.cc:374] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:51.264676 28204 raft_consensus.cc:729] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4704fd423bb4ae4a2d7c7e9ee41002c, State: Initialized, Role: FOLLOWER
I20260808 20:56:51.264719 28204 consensus_queue.cc:260] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.264806 28204 ts_tablet_manager.cc:1428] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:51.265462 28206 tablet_bootstrap.cc:492] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: Bootstrap starting.
I20260808 20:56:51.265897 28206 tablet_bootstrap.cc:654] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:51.266534 28206 tablet_bootstrap.cc:492] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: No bootstrap required, opened a new log
I20260808 20:56:51.266583 28206 ts_tablet_manager.cc:1397] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:51.266729 28206 raft_consensus.cc:348] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.266794 28206 raft_consensus.cc:374] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:51.266814 28206 raft_consensus.cc:729] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c5b28b4acc4cb58e89e035db0cacfe, State: Initialized, Role: FOLLOWER
I20260808 20:56:51.266853 28206 consensus_queue.cc:260] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.266932 28206 ts_tablet_manager.cc:1428] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:51.267362 28205 tablet_bootstrap.cc:492] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: No bootstrap required, opened a new log
I20260808 20:56:51.267414 28205 ts_tablet_manager.cc:1397] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 20:56:51.267580 28205 raft_consensus.cc:348] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.267649 28205 raft_consensus.cc:374] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:51.267670 28205 raft_consensus.cc:729] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d06fbc23484fab9905b6e5172142de, State: Initialized, Role: FOLLOWER
I20260808 20:56:51.267719 28205 consensus_queue.cc:260] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.267807 28205 ts_tablet_manager.cc:1428] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:51.649784 28219 raft_consensus.cc:482] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:51.649904 28219 raft_consensus.cc:504] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.650136 28219 leader_election.cc:288] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130:45279)
I20260808 20:56:51.650432 27698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420dd629a9c84b26ac6526d5347438cb" candidate_uuid: "64d06fbc23484fab9905b6e5172142de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" is_pre_election: true
I20260808 20:56:51.650434 27901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420dd629a9c84b26ac6526d5347438cb" candidate_uuid: "64d06fbc23484fab9905b6e5172142de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" is_pre_election: true
I20260808 20:56:51.650555 27698 raft_consensus.cc:2454] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d06fbc23484fab9905b6e5172142de in term 0.
I20260808 20:56:51.650584 27901 raft_consensus.cc:2454] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d06fbc23484fab9905b6e5172142de in term 0.
I20260808 20:56:51.650720 27995 leader_election.cc:302] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d06fbc23484fab9905b6e5172142de, e4704fd423bb4ae4a2d7c7e9ee41002c; no voters:
I20260808 20:56:51.650858 28219 raft_consensus.cc:2781] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:51.650914 28219 raft_consensus.cc:482] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:51.650929 28219 raft_consensus.cc:3037] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.651664 28219 raft_consensus.cc:504] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.651809 28219 leader_election.cc:288] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [CANDIDATE]: Term 1 election: Requested vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130:45279)
I20260808 20:56:51.651978 27901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420dd629a9c84b26ac6526d5347438cb" candidate_uuid: "64d06fbc23484fab9905b6e5172142de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c"
I20260808 20:56:51.651960 27698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420dd629a9c84b26ac6526d5347438cb" candidate_uuid: "64d06fbc23484fab9905b6e5172142de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe"
I20260808 20:56:51.652050 27901 raft_consensus.cc:3037] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.652050 27698 raft_consensus.cc:3037] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.652911 27901 raft_consensus.cc:2454] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d06fbc23484fab9905b6e5172142de in term 1.
I20260808 20:56:51.653165 27995 leader_election.cc:302] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d06fbc23484fab9905b6e5172142de, e4704fd423bb4ae4a2d7c7e9ee41002c; no voters:
I20260808 20:56:51.653227 27698 raft_consensus.cc:2454] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d06fbc23484fab9905b6e5172142de in term 1.
I20260808 20:56:51.653292 28219 raft_consensus.cc:2781] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:51.653388 28219 raft_consensus.cc:686] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 1 LEADER]: Becoming Leader. State: Replica: 64d06fbc23484fab9905b6e5172142de, State: Running, Role: LEADER
I20260808 20:56:51.653460 28219 consensus_queue.cc:237] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.654294 27459 catalog_manager.cc:5495] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de reported cstate change: term changed from 0 to 1, leader changed from <none> to 64d06fbc23484fab9905b6e5172142de (127.15.232.131). New cstate: current_term: 1 leader_uuid: "64d06fbc23484fab9905b6e5172142de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } health_report { overall_health: HEALTHY } } }
I20260808 20:56:51.708555 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.706042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.15.232.131:44185)
0808 20:56:51.706150 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.706153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.706163 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:51.706193 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.706194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.706209 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.706310 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:51.706510 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.707012 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.707825 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.708042 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.708110 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.708232 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.708269 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.708309 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.708358 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.708408 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
I20260808 20:56:51.709506 27901 raft_consensus.cc:1261] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Refusing update from remote peer 64d06fbc23484fab9905b6e5172142de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:51.709553 27698 raft_consensus.cc:1261] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Refusing update from remote peer 64d06fbc23484fab9905b6e5172142de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:51.709772 28248 consensus_queue.cc:1035] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:51.709874 28219 consensus_queue.cc:1035] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [LEADER]: Connected to new peer: Peer: permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:51.712610 27462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:51.712709 27462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:51.713361 27462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a: Sending DeleteTablet for 3 replicas of tablet 420dd629a9c84b26ac6526d5347438cb
I20260808 20:56:51.713711 27872 tablet_service.cc:1514] Processing DeleteTablet for tablet 420dd629a9c84b26ac6526d5347438cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:51 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:46252
I20260808 20:56:51.713719 28073 tablet_service.cc:1514] Processing DeleteTablet for tablet 420dd629a9c84b26ac6526d5347438cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:51 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34896
I20260808 20:56:51.713732 27674 tablet_service.cc:1514] Processing DeleteTablet for tablet 420dd629a9c84b26ac6526d5347438cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:51 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34224
I20260808 20:56:51.713816 28241 tablet_replica.cc:330] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: stopping tablet replica
I20260808 20:56:51.713862 28240 tablet_replica.cc:330] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: stopping tablet replica
I20260808 20:56:51.713867 28241 raft_consensus.cc:2229] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:51.713917 28240 raft_consensus.cc:2229] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:51.713929 28241 raft_consensus.cc:2258] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:51.713837 27462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 20:56:51.713960 28240 raft_consensus.cc:2258] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 20:56:51.714012 27462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:56:51.714043 28321 tablet_replica.cc:330] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: stopping tablet replica
I20260808 20:56:51.714095 28321 raft_consensus.cc:2229] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:51.714149 28321 raft_consensus.cc:2258] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:51.714265 28241 ts_tablet_manager.cc:1905] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:51.714440 28240 ts_tablet_manager.cc:1905] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:51.714460 28321 ts_tablet_manager.cc:1905] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:51.715852 28241 ts_tablet_manager.cc:1918] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:51.715880 28240 ts_tablet_manager.cc:1918] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:51.715916 28241 log.cc:1196] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/420dd629a9c84b26ac6526d5347438cb
I20260808 20:56:51.715939 28240 log.cc:1196] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/420dd629a9c84b26ac6526d5347438cb
I20260808 20:56:51.716168 28241 ts_tablet_manager.cc:1939] T 420dd629a9c84b26ac6526d5347438cb P 64d06fbc23484fab9905b6e5172142de: Deleting consensus metadata
I20260808 20:56:51.716163 28321 ts_tablet_manager.cc:1918] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:56:51.716184 28240 ts_tablet_manager.cc:1939] T 420dd629a9c84b26ac6526d5347438cb P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting consensus metadata
I20260808 20:56:51.716214 28321 log.cc:1196] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/420dd629a9c84b26ac6526d5347438cb
I20260808 20:56:51.716434 28321 ts_tablet_manager.cc:1939] T 420dd629a9c84b26ac6526d5347438cb P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting consensus metadata
I20260808 20:56:51.716508 27444 catalog_manager.cc:4841] TS e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130:45279): tablet 420dd629a9c84b26ac6526d5347438cb (table test-table [id=82447e8b12864eeab73ba5bc7d78e07d]) successfully deleted
I20260808 20:56:51.716586 27444 catalog_manager.cc:4841] TS 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185): tablet 420dd629a9c84b26ac6526d5347438cb (table test-table [id=82447e8b12864eeab73ba5bc7d78e07d]) successfully deleted
I20260808 20:56:51.716748 27445 catalog_manager.cc:4841] TS 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431): tablet 420dd629a9c84b26ac6526d5347438cb (table test-table [id=82447e8b12864eeab73ba5bc7d78e07d]) successfully deleted
I20260808 20:56:51.717191 27674 tablet_service.cc:1467] Processing CreateTablet for tablet 979932accc684ea39d4d82b0f32d5171 (DEFAULT_TABLE table=test-table [id=fe2840ed31a2479e9d421c0b22c9af0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:51.717203 27872 tablet_service.cc:1467] Processing CreateTablet for tablet 979932accc684ea39d4d82b0f32d5171 (DEFAULT_TABLE table=test-table [id=fe2840ed31a2479e9d421c0b22c9af0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:51.717235 28073 tablet_service.cc:1467] Processing CreateTablet for tablet 979932accc684ea39d4d82b0f32d5171 (DEFAULT_TABLE table=test-table [id=fe2840ed31a2479e9d421c0b22c9af0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:56:51.717319 27872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 979932accc684ea39d4d82b0f32d5171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:51.717316 27674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 979932accc684ea39d4d82b0f32d5171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:51.717327 28073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 979932accc684ea39d4d82b0f32d5171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:51.718432 28204 tablet_bootstrap.cc:492] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: Bootstrap starting.
I20260808 20:56:51.718489 28206 tablet_bootstrap.cc:492] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe: Bootstrap starting.
I20260808 20:56:51.718828 28206 tablet_bootstrap.cc:654] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:51.718827 28204 tablet_bootstrap.cc:654] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:51.719115 28205 tablet_bootstrap.cc:492] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: Bootstrap starting.
I20260808 20:56:51.719514 28204 tablet_bootstrap.cc:492] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: No bootstrap required, opened a new log
I20260808 20:56:51.719511 28206 tablet_bootstrap.cc:492] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe: No bootstrap required, opened a new log
I20260808 20:56:51.719565 28206 ts_tablet_manager.cc:1397] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:51.719565 28204 ts_tablet_manager.cc:1397] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:51.719599 28205 tablet_bootstrap.cc:654] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:51.719682 28204 raft_consensus.cc:348] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.719710 28206 raft_consensus.cc:348] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.719759 28204 raft_consensus.cc:374] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:51.719769 28206 raft_consensus.cc:374] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:51.719786 28204 raft_consensus.cc:729] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4704fd423bb4ae4a2d7c7e9ee41002c, State: Initialized, Role: FOLLOWER
I20260808 20:56:51.719792 28206 raft_consensus.cc:729] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c5b28b4acc4cb58e89e035db0cacfe, State: Initialized, Role: FOLLOWER
I20260808 20:56:51.719830 28204 consensus_queue.cc:260] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.719838 28206 consensus_queue.cc:260] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.719931 28206 ts_tablet_manager.cc:1428] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:51.719933 28204 ts_tablet_manager.cc:1428] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:51.720289 28205 tablet_bootstrap.cc:492] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: No bootstrap required, opened a new log
I20260808 20:56:51.720345 28205 ts_tablet_manager.cc:1397] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:51.720467 28205 raft_consensus.cc:348] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.720543 28205 raft_consensus.cc:374] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:51.720573 28205 raft_consensus.cc:729] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d06fbc23484fab9905b6e5172142de, State: Initialized, Role: FOLLOWER
I20260808 20:56:51.720636 28205 consensus_queue.cc:260] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.720741 28205 ts_tablet_manager.cc:1428] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:51.919785 28220 raft_consensus.cc:482] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:56:51.919960 28220 raft_consensus.cc:504] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.920193 28220 leader_election.cc:288] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185)
I20260808 20:56:51.920559 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979932accc684ea39d4d82b0f32d5171" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d06fbc23484fab9905b6e5172142de" is_pre_election: true
I20260808 20:56:51.920560 27698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979932accc684ea39d4d82b0f32d5171" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" is_pre_election: true
I20260808 20:56:51.920691 27698 raft_consensus.cc:2454] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 0.
I20260808 20:56:51.920698 28100 raft_consensus.cc:2454] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 0.
I20260808 20:56:51.920892 27802 leader_election.cc:302] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48c5b28b4acc4cb58e89e035db0cacfe, e4704fd423bb4ae4a2d7c7e9ee41002c; no voters:
I20260808 20:56:51.921012 28220 raft_consensus.cc:2781] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:56:51.921060 28220 raft_consensus.cc:482] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:56:51.921082 28220 raft_consensus.cc:3037] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.921788 28220 raft_consensus.cc:504] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.921967 28220 leader_election.cc:288] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 election: Requested vote from peers 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431), 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185)
I20260808 20:56:51.922091 27698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979932accc684ea39d4d82b0f32d5171" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c5b28b4acc4cb58e89e035db0cacfe"
I20260808 20:56:51.922190 27698 raft_consensus.cc:3037] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.922159 28100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "979932accc684ea39d4d82b0f32d5171" candidate_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d06fbc23484fab9905b6e5172142de"
I20260808 20:56:51.922243 28100 raft_consensus.cc:3037] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:51.922859 28100 raft_consensus.cc:2454] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 1.
I20260808 20:56:51.923077 27799 leader_election.cc:302] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d06fbc23484fab9905b6e5172142de, e4704fd423bb4ae4a2d7c7e9ee41002c; no voters:
I20260808 20:56:51.923185 28220 raft_consensus.cc:2781] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:51.923204 27698 raft_consensus.cc:2454] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4704fd423bb4ae4a2d7c7e9ee41002c in term 1.
I20260808 20:56:51.923301 28220 raft_consensus.cc:686] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 LEADER]: Becoming Leader. State: Replica: e4704fd423bb4ae4a2d7c7e9ee41002c, State: Running, Role: LEADER
I20260808 20:56:51.923368 28220 consensus_queue.cc:237] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } }
I20260808 20:56:51.924057 27461 catalog_manager.cc:5495] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c reported cstate change: term changed from 0 to 1, leader changed from <none> to e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130). New cstate: current_term: 1 leader_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4704fd423bb4ae4a2d7c7e9ee41002c" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 45279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 } health_report { overall_health: UNKNOWN } } }
I20260808 20:56:51.951186 28330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.948848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.15.232.130:45279)
0808 20:56:51.949014 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.949016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.949024 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:56:51.949045 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.949046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.949058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.949146 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:51.949289 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.949778 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.950515 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.950721 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.950779 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.950908 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.950938 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.950977 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.951031 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.951051 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260808 20:56:51.955994 28330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.953790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:34493 (local address 127.15.232.130:45279)
0808 20:56:51.953830 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:51.953832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:51.953891 (+ 59us) server_negotiation.cc:407] Connection header received
0808 20:56:51.953958 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:51.953959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:51.953969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:51.954026 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:56:51.954142 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.954598 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.955289 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.955519 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.955582 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.955748 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.955777 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.955830 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:51.955889 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 20:56:51.955933 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 20:56:51.956003 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.953688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:45279 (local address 127.15.232.130:34493)
0808 20:56:51.953830 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:51.953841 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:51.953903 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:51.954039 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:51.954040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:51.954042 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 20:56:51.954080 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.954081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.954666 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.954666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.955202 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:51.955203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:51.955582 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:51.955583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:51.955625 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:51.955629 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 20:56:51.955842 (+ 213us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 20:56:51.955843 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 20:56:51.955880 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 20:56:51.955940 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":18,"thread_start_us":94,"threads_started":1}
I20260808 20:56:51.958073 28100 raft_consensus.cc:1261] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Refusing update from remote peer e4704fd423bb4ae4a2d7c7e9ee41002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:51.958348 27698 raft_consensus.cc:1261] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Refusing update from remote peer e4704fd423bb4ae4a2d7c7e9ee41002c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:56:51.958312 28328 consensus_queue.cc:1035] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "64d06fbc23484fab9905b6e5172142de" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:51.958541 28328 consensus_queue.cc:1035] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [LEADER]: Connected to new peer: Peer: permanent_uuid: "48c5b28b4acc4cb58e89e035db0cacfe" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 37431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:56:51.969063 27461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:51.969172 27461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:56:51.969866 27461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8466e036ad72419486c0b240afd5c49a: Sending DeleteTablet for 3 replicas of tablet 979932accc684ea39d4d82b0f32d5171
I20260808 20:56:51.970151 27872 tablet_service.cc:1514] Processing DeleteTablet for tablet 979932accc684ea39d4d82b0f32d5171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:51 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:46252
I20260808 20:56:51.970150 27674 tablet_service.cc:1514] Processing DeleteTablet for tablet 979932accc684ea39d4d82b0f32d5171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:51 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34224
I20260808 20:56:51.970156 28073 tablet_service.cc:1514] Processing DeleteTablet for tablet 979932accc684ea39d4d82b0f32d5171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:56:51 UTC) from {username='oryx', principal='oryx/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:34896
I20260808 20:56:51.970244 28240 tablet_replica.cc:330] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: stopping tablet replica
I20260808 20:56:51.970260 28321 tablet_replica.cc:330] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe: stopping tablet replica
I20260808 20:56:51.970279 28241 tablet_replica.cc:330] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: stopping tablet replica
I20260808 20:56:51.970300 28240 raft_consensus.cc:2229] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:56:51.970319 28321 raft_consensus.cc:2229] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:51.970324 28241 raft_consensus.cc:2229] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:56:51.970342 28240 raft_consensus.cc:2258] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:51.970355 28241 raft_consensus.cc:2258] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:51.970355 28321 raft_consensus.cc:2258] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:56:51.970713 28321 ts_tablet_manager.cc:1905] T 979932accc684ea39d4d82b0f32d5171 P 48c5b28b4acc4cb58e89e035db0cacfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:51.970733 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 27571
I20260808 20:56:51.970762 28240 ts_tablet_manager.cc:1905] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:51.972484 28240 ts_tablet_manager.cc:1918] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:51.972553 28240 log.cc:1196] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/979932accc684ea39d4d82b0f32d5171
I20260808 20:56:51.972754 28240 ts_tablet_manager.cc:1939] T 979932accc684ea39d4d82b0f32d5171 P e4704fd423bb4ae4a2d7c7e9ee41002c: Deleting consensus metadata
I20260808 20:56:51.973976 27444 catalog_manager.cc:4841] TS e4704fd423bb4ae4a2d7c7e9ee41002c (127.15.232.130:45279): tablet 979932accc684ea39d4d82b0f32d5171 (table test-table [id=fe2840ed31a2479e9d421c0b22c9af0e]) successfully deleted
I20260808 20:56:51.979110 28241 ts_tablet_manager.cc:1905] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:56:51.981122 28241 ts_tablet_manager.cc:1918] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 20:56:51.981201 28241 log.cc:1196] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/979932accc684ea39d4d82b0f32d5171
I20260808 20:56:51.981424 28241 ts_tablet_manager.cc:1939] T 979932accc684ea39d4d82b0f32d5171 P 64d06fbc23484fab9905b6e5172142de: Deleting consensus metadata
I20260808 20:56:51.981863 27444 catalog_manager.cc:4841] TS 64d06fbc23484fab9905b6e5172142de (127.15.232.131:44185): tablet 979932accc684ea39d4d82b0f32d5171 (table test-table [id=fe2840ed31a2479e9d421c0b22c9af0e]) successfully deleted
W20260808 20:56:51.983997 27445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:37431 (error 108)
I20260808 20:56:51.984627 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:51.984431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:37431 (local address 127.0.0.1:34236)
0808 20:56:51.984534 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:51.984580 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:37431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":64,"threads_started":1}
I20260808 20:56:51.984673 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 27778
W20260808 20:56:51.984788 27445 catalog_manager.cc:4584] TS 48c5b28b4acc4cb58e89e035db0cacfe (127.15.232.129:37431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 979932accc684ea39d4d82b0f32d5171: Network error: Client connection negotiation failed: client connection to 127.15.232.129:37431: connect: Connection refused (error 111)
I20260808 20:56:51.999850 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 27971
I20260808 20:56:52.016158 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 27418
2026-08-08T20:56:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3547 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:52 dist-test-slave-c12x krb5kdc[28345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:52 dist-test-slave-c12x krb5kdc[28345](info): set up 2 sockets
Aug 08 20:56:52 dist-test-slave-c12x krb5kdc[28345](info): commencing operation
krb5kdc: starting...
W20260808 20:56:53.257215 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:53 dist-test-slave-c12x krb5kdc[28345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:53.390681 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33621
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:33621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:53.477769 28365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:53.478034 28365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:53.478075 28365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:53.479283 28365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:53.479360 28365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:53.479405 28365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:53.479460 28365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:53.479489 28365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:53.481067 28365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:33621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:53.481361 28365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:53.481617 28365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:53.486785 28378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:53.486791 28374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:53.486812 28375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:53.487083 28365 server_base.cc:1034] running on GCE node
I20260808 20:56:53.487325 28365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:53.487650 28365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:53.488828 28365 hybrid_clock.cc:648] HybridClock initialized: now 1786222613488814 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T20:56:53Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1477 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:56:53 dist-test-slave-c12x krb5kdc[28387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:56:53 dist-test-slave-c12x krb5kdc[28387](info): set up 2 sockets
Aug 08 20:56:53 dist-test-slave-c12x krb5kdc[28387](info): commencing operation
krb5kdc: starting...
W20260808 20:56:54.727236 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:56:54 dist-test-slave-c12x krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:56:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:56:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:56:54.862846 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:41527
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:45169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:41527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:54.950563 28405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:54.950789 28405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:54.950814 28405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:54.952045 28405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:54.952091 28405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:54.952105 28405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:54.952118 28405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:54.952131 28405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:54.953729 28405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:41527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:41527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:54.954021 28405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:54.954267 28405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:54.959544 28414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:54.959674 28405 server_base.cc:1034] running on GCE node
W20260808 20:56:54.959481 28412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:54.959481 28411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:54.960108 28405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:54.960427 28405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:54.961596 28405 hybrid_clock.cc:648] HybridClock initialized: now 1786222614961558 us; error 50 us; skew 500 ppm
Aug 08 20:56:54 dist-test-slave-c12x krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222614, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:54.964214 28405 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:54.964644 28405 webserver.cc:460] Webserver started at http://127.15.232.190:41901/ using document root <none> and password file <none>
I20260808 20:56:54.964896 28405 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:54.964943 28405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:54.965096 28405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:54.966190 28405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8"
format_stamp: "Formatted at 2026-08-08 20:56:54 on dist-test-slave-c12x"
server_key: "50337c65462783cbd6583246c38f0bc3"
server_key_iv: "0a4db5f388cc12f640a5c22a6fb9ba1b"
server_key_version: "encryptionkey@0"
I20260808 20:56:54.966701 28405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8"
format_stamp: "Formatted at 2026-08-08 20:56:54 on dist-test-slave-c12x"
server_key: "50337c65462783cbd6583246c38f0bc3"
server_key_iv: "0a4db5f388cc12f640a5c22a6fb9ba1b"
server_key_version: "encryptionkey@0"
I20260808 20:56:54.968259 28405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:56:54.969072 28422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:54.969260 28405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:54.969372 28405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8"
format_stamp: "Formatted at 2026-08-08 20:56:54 on dist-test-slave-c12x"
server_key: "50337c65462783cbd6583246c38f0bc3"
server_key_iv: "0a4db5f388cc12f640a5c22a6fb9ba1b"
server_key_version: "encryptionkey@0"
I20260808 20:56:54.969453 28405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:54.989688 28405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:54.992295 28405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:54.992487 28405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:54.997352 28405 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:41527
I20260808 20:56:54.997386 28499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:41527 every 8 connection(s)
I20260808 20:56:54.997810 28405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:54.998781 28500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:56:54.999339 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 28405
I20260808 20:56:54.999501 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:56:54.999707 16290 external_mini_cluster.cc:1428] Setting key 7a19564f6c0da9e1fc72186ce9a521e9
I20260808 20:56:55.002385 28500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: Bootstrap starting.
I20260808 20:56:55.003123 28500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: Neither blocks nor log segments found. Creating new log.
I20260808 20:56:55.003468 28500 log.cc:824] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:55.004276 28500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: No bootstrap required, opened a new log
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:55.007051 28500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } }
I20260808 20:56:55.007202 28500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:56:55.007226 28500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ecc4a5cbad42cf8ddfd7238ab83db8, State: Initialized, Role: FOLLOWER
I20260808 20:56:55.007408 28500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } }
I20260808 20:56:55.007524 28500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:55.007567 28500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:55.007616 28500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:56:55.008703 28500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } }
I20260808 20:56:55.008934 28500 leader_election.cc:302] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12ecc4a5cbad42cf8ddfd7238ab83db8; no voters:
I20260808 20:56:55.009191 28500 leader_election.cc:288] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:56:55.009230 28507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:56:55.009342 28507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 1 LEADER]: Becoming Leader. State: Replica: 12ecc4a5cbad42cf8ddfd7238ab83db8, State: Running, Role: LEADER
I20260808 20:56:55.009455 28507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } }
I20260808 20:56:55.009524 28500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:55.009800 28507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } } }
I20260808 20:56:55.009932 28508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12ecc4a5cbad42cf8ddfd7238ab83db8. Latest consensus state: current_term: 1 leader_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } } }
I20260808 20:56:55.009979 28507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:55.010115 28508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:55.010362 28519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:55.011000 28519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:55.011646 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.000319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.15.232.190:41527)
0808 20:56:55.000504 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:55.000512 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:55.000562 (+ 50us) server_negotiation.cc:407] Connection header received
0808 20:56:55.000977 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:55.000984 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:55.001033 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:55.001152 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:55.001998 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.002980 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.004059 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.004529 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.004615 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.007421 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:55.007425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:55.007425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:55.007489 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:55.009795 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:55.010235 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:55.010238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:55.010239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:55.010351 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:55.010595 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:55.010596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:55.010597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:55.010938 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:55.011083 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:55.011205 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 20:56:55.011353 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260808 20:56:55.012869 28519 catalog_manager.cc:1347] Generated new cluster ID: 041fa510e412412ebcc30eed5f18af0f
I20260808 20:56:55.012923 28519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:55.023093 28519 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:56:55.023823 28519 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:55.036609 28519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: Generated new TSK 0
I20260808 20:56:55.036864 28519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:56:55.074764 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41527
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:45169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:55.161787 28541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:55.162004 28541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:55.162030 28541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:55.163271 28541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:55.163319 28541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:55.163349 28541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:55.164842 28541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:55.165071 28541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:55.165350 28541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:55.170472 28550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:55.170588 28546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:55.170485 28547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:55.170823 28541 server_base.cc:1034] running on GCE node
I20260808 20:56:55.171105 28541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:55.171371 28541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:55.172639 28541 hybrid_clock.cc:648] HybridClock initialized: now 1786222615172625 us; error 31 us; skew 500 ppm
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222615, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:55.175356 28541 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:56:55.175840 28541 webserver.cc:460] Webserver started at http://127.15.232.129:33077/ using document root <none> and password file <none>
I20260808 20:56:55.176092 28541 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:55.176163 28541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:55.176309 28541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:55.177394 28541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "c06c945b6a2b4a8492e263145b63113f"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "732d383d7ee620c8b36db741fa62b25e"
server_key_iv: "d2290cdd64aed37c67c69f4a00dbe624"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.177830 28541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "c06c945b6a2b4a8492e263145b63113f"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "732d383d7ee620c8b36db741fa62b25e"
server_key_iv: "d2290cdd64aed37c67c69f4a00dbe624"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.179248 28541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 20:56:55.180110 28561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.180291 28541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:56:55.180395 28541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "c06c945b6a2b4a8492e263145b63113f"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "732d383d7ee620c8b36db741fa62b25e"
server_key_iv: "d2290cdd64aed37c67c69f4a00dbe624"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.180467 28541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:55.189354 28541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:55.191707 28541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:55.191875 28541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:55.192114 28541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:55.192430 28541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:55.192487 28541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.192528 28541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:55.192552 28541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.198503 28541 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:34571
I20260808 20:56:55.198544 28713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:34571 every 8 connection(s)
I20260808 20:56:55.198913 28541 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:55.201116 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 28541
I20260808 20:56:55.201218 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:56:55.201443 16290 external_mini_cluster.cc:1428] Setting key 5907121754cc0ae299479d6bd0489874
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222615, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:55.210224 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.199784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:37579 (local address 127.15.232.190:41527)
0808 20:56:55.199861 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:55.199863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:55.200090 (+ 227us) server_negotiation.cc:407] Connection header received
0808 20:56:55.200618 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:55.200619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:55.200630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:55.200686 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:55.201871 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.202619 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.203785 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.204128 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.204220 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.207071 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:55.207073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:55.207073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:55.207082 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:55.208644 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:55.209140 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:55.209141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:55.209142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:55.209205 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:55.209534 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:55.209535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:55.209536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:55.209656 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:55.209737 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:55.210026 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 20:56:55.210125 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 20:56:55.210443 28718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.199741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:37579)
0808 20:56:55.199946 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:55.199984 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:55.200514 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:55.200742 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:55.200745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:55.201442 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:55.201743 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.201750 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.202717 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.202718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.203630 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.203633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.204243 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.204245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.204410 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.205047 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:55.205059 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:55.206953 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:55.208757 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:55.208765 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:55.208768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:55.209015 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:55.209284 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:55.209285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:55.209285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:55.209414 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:55.209727 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:55.209734 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:55.209889 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 20:56:55.210013 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 20:56:55.210191 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
I20260808 20:56:55.210999 28715 heartbeater.cc:347] Connected to a master server at 127.15.232.190:41527
I20260808 20:56:55.211103 28715 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:55.211313 28715 heartbeater.cc:511] Master 127.15.232.190:41527 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
I20260808 20:56:55.211951 28455 ts_manager.cc:194] Registered new tserver with Master: c06c945b6a2b4a8492e263145b63113f (127.15.232.129:34571)
I20260808 20:56:55.212646 28455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:37579
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:56:55.231166 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41527
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:45169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:55.319120 28722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:55.319344 28722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:55.319379 28722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:55.320669 28722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:55.320717 28722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:55.320744 28722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:55.322227 28722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:55.322438 28722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:55.322631 28722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:55.328274 28732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:55.328408 28734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:55.328329 28722 server_base.cc:1034] running on GCE node
W20260808 20:56:55.328558 28731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:55.328953 28722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:55.329253 28722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:55.330435 28722 hybrid_clock.cc:648] HybridClock initialized: now 1786222615330411 us; error 60 us; skew 500 ppm
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222615, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:55.333182 28722 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:56:55.333750 28722 webserver.cc:460] Webserver started at http://127.15.232.130:37581/ using document root <none> and password file <none>
I20260808 20:56:55.333997 28722 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:55.334069 28722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:55.334223 28722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:55.335398 28722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "5a744dd176dd4c318e67925e853d1dfb"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "1fc13a5e30c9dbec39288d344f31de66"
server_key_iv: "1842c853e88cfe5c7bc9c432787b8104"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.335855 28722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "5a744dd176dd4c318e67925e853d1dfb"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "1fc13a5e30c9dbec39288d344f31de66"
server_key_iv: "1842c853e88cfe5c7bc9c432787b8104"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.337371 28722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:55.338217 28744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.338413 28722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:55.338514 28722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "5a744dd176dd4c318e67925e853d1dfb"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "1fc13a5e30c9dbec39288d344f31de66"
server_key_iv: "1842c853e88cfe5c7bc9c432787b8104"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.338595 28722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:55.351704 28722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:55.353961 28722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:55.354151 28722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:55.354416 28722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:55.354760 28722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:55.354794 28722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.354817 28722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:55.354864 28722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.361682 28722 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:33209
I20260808 20:56:55.361773 28909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:33209 every 8 connection(s)
I20260808 20:56:55.362114 28722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:56:55.367776 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 28722
I20260808 20:56:55.367873 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:56:55.368083 16290 external_mini_cluster.cc:1428] Setting key 35eb10741ae3f1c61302a71e651bf44c
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222615, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:55.372817 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.362904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:41559)
0808 20:56:55.363074 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:55.363119 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:55.363661 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:55.364037 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:55.364040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:55.364637 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:55.364843 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.364848 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.365594 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.365594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.366395 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.366397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.366837 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.366839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.366994 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.367604 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:55.367612 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:55.369417 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:55.371294 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:55.371301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:55.371303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:55.371639 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:55.371886 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:55.371886 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:55.371887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:55.371997 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:55.372303 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:55.372308 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:55.372432 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 20:56:55.372514 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 20:56:55.372649 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260808 20:56:55.372815 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.362967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:41559 (local address 127.15.232.190:41527)
0808 20:56:55.363062 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:55.363065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:55.363214 (+ 149us) server_negotiation.cc:407] Connection header received
0808 20:56:55.363823 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:55.363824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:55.363841 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:55.363942 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:55.364977 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.365496 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.366488 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.366720 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.366782 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.369563 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:55.369565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:55.369565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:55.369572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:55.371200 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:55.371730 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:55.371731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:55.371732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:55.371803 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:55.372083 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:55.372084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:55.372084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:55.372236 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:55.372305 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:55.372581 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 20:56:55.372674 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 20:56:55.373399 28910 heartbeater.cc:347] Connected to a master server at 127.15.232.190:41527
I20260808 20:56:55.373509 28910 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:55.373723 28910 heartbeater.cc:511] Master 127.15.232.190:41527 requested a full tablet report, sending...
I20260808 20:56:55.374213 28455 ts_manager.cc:194] Registered new tserver with Master: 5a744dd176dd4c318e67925e853d1dfb (127.15.232.130:33209)
I20260808 20:56:55.374754 28455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:41559
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:56:55.396026 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41527
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:45169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:55.481540 28915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:55.481753 28915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:55.481779 28915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:55.483101 28915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:55.483147 28915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:55.483176 28915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:55.484756 28915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:55.485028 28915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:55.485270 28915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:55.491103 28925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:55.491174 28922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:55.491307 28915 server_base.cc:1034] running on GCE node
W20260808 20:56:55.491174 28923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:55.491644 28915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:55.491943 28915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:55.493121 28915 hybrid_clock.cc:648] HybridClock initialized: now 1786222615493110 us; error 26 us; skew 500 ppm
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222615, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:55.495714 28915 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:56:55.496220 28915 webserver.cc:460] Webserver started at http://127.15.232.131:45263/ using document root <none> and password file <none>
I20260808 20:56:55.496469 28915 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:55.496541 28915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:55.496701 28915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:56:55.497956 28915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "67d187f1dbf540bfb50348bbebc5127c"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "f0807ad9deadcaab7056b435d421d7a7"
server_key_iv: "d8a138bc7c0d505c3ece686f507206c5"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.498379 28915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "67d187f1dbf540bfb50348bbebc5127c"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "f0807ad9deadcaab7056b435d421d7a7"
server_key_iv: "d8a138bc7c0d505c3ece686f507206c5"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.499962 28915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:56:55.501020 28935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.501294 28915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:55.501437 28915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "67d187f1dbf540bfb50348bbebc5127c"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "f0807ad9deadcaab7056b435d421d7a7"
server_key_iv: "d8a138bc7c0d505c3ece686f507206c5"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.501530 28915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:55.513265 28915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:55.515828 28915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:55.516021 28915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:55.516284 28915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:55.516613 28915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:55.516669 28915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.516711 28915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:55.516741 28915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.523037 28915 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:42653
I20260808 20:56:55.523085 29107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:42653 every 8 connection(s)
I20260808 20:56:55.523424 28915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222615, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:56:55.532156 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 28915
I20260808 20:56:55.532274 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:56:55.532508 16290 external_mini_cluster.cc:1428] Setting key daaa50f3f487e0815a7c9e1ffe0bfd8d
I20260808 20:56:55.534138 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.524259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:45533 (local address 127.15.232.190:41527)
0808 20:56:55.524337 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:55.524340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:55.524573 (+ 233us) server_negotiation.cc:407] Connection header received
0808 20:56:55.525073 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:55.525074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:55.525085 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:55.525141 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:55.526427 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.527144 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.528252 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.528489 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.528571 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.530911 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:55.530912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:55.530913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:55.530921 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:55.532607 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:55.533091 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:55.533093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:55.533093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:55.533161 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:55.533429 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:55.533430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:55.533430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:55.533565 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:55.533641 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:55.533927 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 20:56:55.534039 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 20:56:55.534358 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.524220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:45533)
0808 20:56:55.524442 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:55.524485 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:55.524966 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:55.525223 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:55.525227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:55.526029 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:55.526327 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.526334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.527245 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.527246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.528154 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.528157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.528581 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.528582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.528734 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.529332 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:55.529341 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:55.530797 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:56:55.532706 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:55.532713 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:55.532715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:55.532946 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:55.533242 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:56:55.533243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:56:55.533243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:56:55.533355 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:56:55.533642 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:56:55.533650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:56:55.533788 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 20:56:55.533905 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 20:56:55.534086 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":120,"threads_started":1}
I20260808 20:56:55.534844 29110 heartbeater.cc:347] Connected to a master server at 127.15.232.190:41527
I20260808 20:56:55.534953 29110 heartbeater.cc:464] Registering TS with master...
I20260808 20:56:55.535136 29110 heartbeater.cc:511] Master 127.15.232.190:41527 requested a full tablet report, sending...
I20260808 20:56:55.535588 28455 ts_manager.cc:194] Registered new tserver with Master: 67d187f1dbf540bfb50348bbebc5127c (127.15.232.131:42653)
I20260808 20:56:55.535977 28455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:45533
I20260808 20:56:55.544230 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab.
Entry for principal oryx/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab.
WARNING: no policy specified for oryx/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab.
Entry for principal oryx/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab.
WARNING: no policy specified for oryx/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab.
Entry for principal oryx/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab.
I20260808 20:56:55.638288 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 28541
I20260808 20:56:55.647212 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 28722
I20260808 20:56:55.659524 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 28915
I20260808 20:56:55.673166 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 28405
I20260808 20:56:55.684664 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:41527
--webserver_interface=127.15.232.190
--webserver_port=41901
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:45169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:41527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:56:55.771611 29120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:55.771840 29120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:55.771874 29120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:55.773000 29120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:56:55.773046 29120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:55.773062 29120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:55.773077 29120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:56:55.773090 29120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:56:55.774612 29120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:41527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:41527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=41901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:55.774844 29120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:55.775045 29120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:55.781306 29128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:55.781306 29126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:55.781347 29120 server_base.cc:1034] running on GCE node
W20260808 20:56:55.781297 29134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:55.781956 29120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:55.782311 29120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:55.783551 29120 hybrid_clock.cc:648] HybridClock initialized: now 1786222615783527 us; error 43 us; skew 500 ppm
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222615, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:55.787110 29120 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:56:55.788095 29120 webserver.cc:460] Webserver started at http://127.15.232.190:41901/ using document root <none> and password file <none>
I20260808 20:56:55.788568 29120 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:55.788693 29120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:55.790213 29120 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:55.791033 29144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.791257 29120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:55.791355 29120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8"
format_stamp: "Formatted at 2026-08-08 20:56:54 on dist-test-slave-c12x"
server_key: "50337c65462783cbd6583246c38f0bc3"
server_key_iv: "0a4db5f388cc12f640a5c22a6fb9ba1b"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.791841 29120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:55.807092 29120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:55.809674 29120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:55.809800 29120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:55.813793 29120 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:41527
I20260808 20:56:55.813884 29223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:41527 every 8 connection(s)
I20260808 20:56:55.814262 29120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:56:55.815997 29224 sys_catalog.cc:263] Verifying existing consensus state
I20260808 20:56:55.816695 29224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: Bootstrap starting.
I20260808 20:56:55.820992 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 29120
I20260808 20:56:55.821372 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:34571
--local_ip_for_outbound_sockets=127.15.232.129
--tserver_master_addrs=127.15.232.190:41527
--webserver_port=33077
--webserver_interface=127.15.232.129
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:45169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.232.129
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:56:55.825947 29224 log.cc:824] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: Log is configured to *not* fsync() on all Append() calls
I20260808 20:56:55.827857 29224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:56:55.828176 29224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: Bootstrap complete.
I20260808 20:56:55.830560 29224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } }
I20260808 20:56:55.830791 29224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ecc4a5cbad42cf8ddfd7238ab83db8, State: Initialized, Role: FOLLOWER
I20260808 20:56:55.830964 29224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } }
I20260808 20:56:55.831063 29224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:56:55.831120 29224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:56:55.831174 29224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 1 FOLLOWER]: Advancing to term 2
I20260808 20:56:55.832290 29224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } }
I20260808 20:56:55.832433 29224 leader_election.cc:302] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12ecc4a5cbad42cf8ddfd7238ab83db8; no voters:
I20260808 20:56:55.832664 29224 leader_election.cc:288] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 20:56:55.832700 29230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 20:56:55.832816 29230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [term 2 LEADER]: Becoming Leader. State: Replica: 12ecc4a5cbad42cf8ddfd7238ab83db8, State: Running, Role: LEADER
I20260808 20:56:55.832950 29224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:56:55.832945 29230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } }
I20260808 20:56:55.833629 29230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12ecc4a5cbad42cf8ddfd7238ab83db8. Latest consensus state: current_term: 2 leader_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } } }
I20260808 20:56:55.833837 29230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:55.833920 29231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12ecc4a5cbad42cf8ddfd7238ab83db8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41527 } } }
I20260808 20:56:55.834015 29231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8 [sys.catalog]: This master's current role is: LEADER
I20260808 20:56:55.834187 29241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:56:55.834730 29241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:56:55.835031 29241 catalog_manager.cc:1259] Loaded cluster ID: 041fa510e412412ebcc30eed5f18af0f
I20260808 20:56:55.835109 29241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:56:55.836388 29241 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:56:55.836555 29241 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 12ecc4a5cbad42cf8ddfd7238ab83db8: Loaded TSK: 0
I20260808 20:56:55.836752 29241 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 20:56:55.909286 29226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:55.909540 29226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:55.909585 29226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:55.910770 29226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:55.910843 29226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:55.910899 29226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:56:55.912849 29226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.129.keytab
--principal=oryx/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:34571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=33077
--tserver_master_addrs=127.15.232.190:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:55.913122 29226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:55.913395 29226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:55.919639 29264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:55.919716 29261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:55.919638 29262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:55.920256 29226 server_base.cc:1034] running on GCE node
I20260808 20:56:55.920507 29226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:55.920825 29226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:55.922017 29226 hybrid_clock.cc:648] HybridClock initialized: now 1786222615922002 us; error 36 us; skew 500 ppm
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222615, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:55.924780 29226 init.cc:377] Logged in from keytab as oryx/127.15.232.129@KRBTEST.COM (short username oryx)
I20260808 20:56:55.925303 29226 webserver.cc:460] Webserver started at http://127.15.232.129:33077/ using document root <none> and password file <none>
I20260808 20:56:55.925576 29226 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:55.925637 29226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:55.927109 29226 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:55.927842 29273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.928128 29226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:55.928249 29226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "c06c945b6a2b4a8492e263145b63113f"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "732d383d7ee620c8b36db741fa62b25e"
server_key_iv: "d2290cdd64aed37c67c69f4a00dbe624"
server_key_version: "encryptionkey@0"
I20260808 20:56:55.928699 29226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:55.951314 29226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:55.953770 29226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:55.953959 29226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:55.954222 29226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:55.954550 29226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:55.954605 29226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.954648 29226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:55.954679 29226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:55.961549 29226 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:34571
I20260808 20:56:55.961647 29434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:34571 every 8 connection(s)
I20260808 20:56:55.961977 29226 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:56:55.968261 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 29226
I20260808 20:56:55.968600 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:33209
--local_ip_for_outbound_sockets=127.15.232.130
--tserver_master_addrs=127.15.232.190:41527
--webserver_port=37581
--webserver_interface=127.15.232.130
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:45169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.232.130
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:55.972604 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.962770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:54435)
0808 20:56:55.963035 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:55.963098 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:55.963936 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:55.964417 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:55.964422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:55.965240 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:55.965507 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.965513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.966611 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.966612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.967932 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.967935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.969006 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.969007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.969174 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.969859 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:55.969870 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:55.972387 (+ 2517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":115,"threads_started":1}
W20260808 20:56:55.972858 29435 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:41527 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.232.190:41527: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
W20260808 20:56:55.972994 29441 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:55.962946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:54435 (local address 127.15.232.190:41527)
0808 20:56:55.963171 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:55.963180 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:55.963218 (+ 38us) server_negotiation.cc:407] Connection header received
0808 20:56:55.964170 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:55.964176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:55.964219 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:55.964335 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:55.965651 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.966511 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.968075 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.968778 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.968891 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.972518 (+ 3627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:54435: BlockingRecv error: recv got EOF from 127.15.232.129:54435 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":111,"threads_started":1}
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:55.978075 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.973096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:43409)
0808 20:56:55.973391 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:55.973402 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:55.973474 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:55.973611 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:55.973612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:55.973741 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:55.973788 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.973789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.974434 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.974435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.974936 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.974938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.975311 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.975312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.975383 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.975868 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:55.975874 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:55.977969 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":75,"threads_started":1}
W20260808 20:56:55.978170 29441 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:55.973311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:43409 (local address 127.15.232.190:41527)
0808 20:56:55.973369 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:55.973371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:55.973460 (+ 89us) server_negotiation.cc:407] Connection header received
0808 20:56:55.973536 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:55.973537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:55.973552 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:55.973599 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:55.973874 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.974352 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.975046 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.975236 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.975313 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.978090 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:43409: BlockingRecv error: recv got EOF from 127.15.232.129:43409 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:55 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:55.983770 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:55.978424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:52307)
0808 20:56:55.978480 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:55.978495 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:55.978550 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:55.978710 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:55.978711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:55.978881 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:55.978926 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.978929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.979825 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.979826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.980694 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:55.980698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.981091 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.981092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:55.981159 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.981624 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:55.981630 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:55.983689 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 20:56:55.984059 29435 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 20:56:55.984083 29441 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:55.978467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:52307 (local address 127.15.232.190:41527)
0808 20:56:55.978525 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:55.978527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:55.978535 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:56:55.978626 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:55.978626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:55.978635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:55.978682 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:55.979007 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.979735 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.980812 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:55.981000 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:55.981073 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:55.984022 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:52307: BlockingRecv error: recv got EOF from 127.15.232.129:52307 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 20:56:56.060760 29442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:56.061002 29442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:56.061030 29442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:56.062322 29442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:56.062383 29442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:56.062414 29442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:56:56.064131 29442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.130.keytab
--principal=oryx/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:33209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=37581
--tserver_master_addrs=127.15.232.190:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:56.064436 29442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:56.064710 29442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:56.070485 29449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:56.070508 29452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:56.070535 29442 server_base.cc:1034] running on GCE node
W20260808 20:56:56.070472 29454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:56.070885 29442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:56.071163 29442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:56.072366 29442 hybrid_clock.cc:648] HybridClock initialized: now 1786222616072346 us; error 45 us; skew 500 ppm
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222616, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:56.075179 29442 init.cc:377] Logged in from keytab as oryx/127.15.232.130@KRBTEST.COM (short username oryx)
I20260808 20:56:56.075742 29442 webserver.cc:460] Webserver started at http://127.15.232.130:37581/ using document root <none> and password file <none>
I20260808 20:56:56.076011 29442 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:56.076071 29442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:56.077783 29442 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:56:56.078578 29470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:56.078805 29442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:56.078933 29442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "5a744dd176dd4c318e67925e853d1dfb"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "1fc13a5e30c9dbec39288d344f31de66"
server_key_iv: "1842c853e88cfe5c7bc9c432787b8104"
server_key_version: "encryptionkey@0"
I20260808 20:56:56.079329 29442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:56.108383 29442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:56.110888 29442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:56.111071 29442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:56.111341 29442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:56.111721 29442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:56.111778 29442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:56.111820 29442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:56.111848 29442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:56.118111 29442 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:33209
I20260808 20:56:56.118206 29634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:33209 every 8 connection(s)
I20260808 20:56:56.118566 29442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:56.126334 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 29442
I20260808 20:56:56.126696 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:42653
--local_ip_for_outbound_sockets=127.15.232.131
--tserver_master_addrs=127.15.232.190:41527
--webserver_port=45263
--webserver_interface=127.15.232.131
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:45169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.232.131
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:56:56.126943 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:56.119331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:59759)
0808 20:56:56.119531 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:56.119571 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:56.120092 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:56.120460 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:56.120463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:56.121065 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:56.121269 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.121275 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.122046 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.122047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.122896 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.122899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.123264 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.123265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.123367 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.123980 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:56.123989 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:56.126702 (+ 2713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
W20260808 20:56:56.126945 29441 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:56.119424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:59759 (local address 127.15.232.190:41527)
0808 20:56:56.119520 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:56.119522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:56.119668 (+ 146us) server_negotiation.cc:407] Connection header received
0808 20:56:56.120216 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:56.120218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:56.120237 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:56.120308 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:56.121379 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.121938 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.122986 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.123185 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.123244 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.126872 (+ 3628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:59759: BlockingRecv error: recv got EOF from 127.15.232.130:59759 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 20:56:56.127233 29636 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:41527 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.232.190:41527: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:56.132627 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:56.127596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:58205)
0808 20:56:56.127738 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:56.127751 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:56.127806 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:56.127995 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:56.127996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:56.128144 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:56.128195 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.128197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.128870 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.128872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.129399 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.129401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.129808 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.129809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.129893 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.130391 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:56.130404 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:56.132468 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125}
W20260808 20:56:56.132663 29441 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:56.127668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:58205 (local address 127.15.232.190:41527)
0808 20:56:56.127740 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:56.127742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:56.127788 (+ 46us) server_negotiation.cc:407] Connection header received
0808 20:56:56.127914 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:56.127915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:56.127926 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:56.127990 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:56.128287 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.128775 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.129508 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.129716 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.129808 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.132615 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:58205: BlockingRecv error: recv got EOF from 127.15.232.130:58205 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:56.138159 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:56.133044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:50541)
0808 20:56:56.133106 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:56.133120 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:56.133173 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:56.133339 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:56.133340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:56.133491 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:56.133521 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.133523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.134231 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.134232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.134905 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.134908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.135304 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.135305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.135357 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.135961 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:56.135966 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:56.138050 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 20:56:56.138268 29441 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:56.133132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:50541 (local address 127.15.232.190:41527)
0808 20:56:56.133172 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:56.133174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:56.133184 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:56.133238 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:56.133239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:56.133250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:56.133333 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:56.133623 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.134123 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.135034 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.135228 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.135305 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.138177 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:50541: BlockingRecv error: recv got EOF from 127.15.232.130:50541 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 20:56:56.138532 29636 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 20:56:56.214834 29640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:56:56.215116 29640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:56:56.215160 29640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:56:56.216451 29640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:56:56.216537 29640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:56:56.216589 29640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:56:56.218235 29640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/krb5kdc/oryx_127.15.232.131.keytab
--principal=oryx/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:42653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=45263
--tserver_master_addrs=127.15.232.190:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:56:56.218544 29640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:56:56.218815 29640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:56:56.224390 29650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:56.224483 29648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:56:56.224368 29652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:56:56.224651 29640 server_base.cc:1034] running on GCE node
I20260808 20:56:56.224895 29640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:56:56.225214 29640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:56:56.226392 29640 hybrid_clock.cc:648] HybridClock initialized: now 1786222616226364 us; error 43 us; skew 500 ppm
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222616, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:56:56.229166 29640 init.cc:377] Logged in from keytab as oryx/127.15.232.131@KRBTEST.COM (short username oryx)
I20260808 20:56:56.229578 29640 webserver.cc:460] Webserver started at http://127.15.232.131:45263/ using document root <none> and password file <none>
I20260808 20:56:56.229768 29640 fs_manager.cc:359] Metadata directory not provided
I20260808 20:56:56.229822 29640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:56:56.231321 29640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:56:56.232147 29662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:56.232342 29640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:56:56.232442 29640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "67d187f1dbf540bfb50348bbebc5127c"
format_stamp: "Formatted at 2026-08-08 20:56:55 on dist-test-slave-c12x"
server_key: "f0807ad9deadcaab7056b435d421d7a7"
server_key_iv: "d8a138bc7c0d505c3ece686f507206c5"
server_key_version: "encryptionkey@0"
I20260808 20:56:56.232846 29640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:56:56.247805 29640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:56:56.249992 29640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:56:56.250165 29640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:56:56.250420 29640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:56:56.250738 29640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:56:56.250793 29640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:56.250836 29640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:56:56.250867 29640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:56:56.256925 29640 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:42653
I20260808 20:56:56.256965 29830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:42653 every 8 connection(s)
I20260808 20:56:56.257319 29640 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:56:56.263298 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 29640
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 20:56:56.265141 29441 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:56.258055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:58619 (local address 127.15.232.190:41527)
0808 20:56:56.258128 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:56.258131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:56.258382 (+ 251us) server_negotiation.cc:407] Connection header received
0808 20:56:56.258975 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:56.258977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:56.258989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:56.259036 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:56.260079 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.260587 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.261615 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.261818 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.261899 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.265079 (+ 3180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:58619: BlockingRecv error: recv got EOF from 127.15.232.131:58619 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 20:56:56.265231 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:56.258049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:58619)
0808 20:56:56.258234 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:56.258293 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:56.258846 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:56.259103 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:56.259109 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:56.259746 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:56.259950 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.259956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.260688 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.260689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.261513 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.261516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.261909 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.261910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.262047 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.262631 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:56.262641 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:56.265010 (+ 2369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
W20260808 20:56:56.265472 29831 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:41527 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.232.190:41527: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:56.270973 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:56.265746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:44689)
0808 20:56:56.265852 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:56.265861 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:56.265904 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:56.266031 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:56.266031 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:56.266151 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:56.266197 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.266198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.266975 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.266976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.267702 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.267705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.268210 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.268212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.268285 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.268718 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:56.268724 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:56.270846 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260808 20:56:56.271288 29441 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:56.265806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:44689 (local address 127.15.232.190:41527)
0808 20:56:56.265842 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:56.265844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:56.265896 (+ 52us) server_negotiation.cc:407] Connection header received
0808 20:56:56.265959 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:56.265960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:56.265970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:56.266028 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:56.266266 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.266884 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.267805 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.268029 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.268208 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.270988 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:44689: BlockingRecv error: recv got EOF from 127.15.232.131:44689 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 20:56:56.272311 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:56.264025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58626 (local address 127.15.232.190:41527)
0808 20:56:56.264169 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:56.264172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:56.264182 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:56.264207 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:56.264207 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:56.264220 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:56.264279 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:56.264914 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.265633 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.266705 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.266918 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.267006 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.268455 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:56:56.268462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:56:56.268462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:56:56.268479 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:56:56.270360 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:56.270898 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:56.270902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:56.270902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:56.271038 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:56:56.271308 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:56:56.271309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:56:56.271309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:56:56.271722 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:56:56.271903 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 20:56:56.272035 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 20:56:56.272171 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:56.276712 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:56.272413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:41291)
0808 20:56:56.272450 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:56.272462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:56.272501 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:56.272640 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:56.272640 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:56.272787 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:56.272824 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.272826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.273705 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.273705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.274200 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.274201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.274515 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.274515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.274555 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.274907 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:56.274911 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:56.276660 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 20:56:56.276919 29838 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:56.272508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:41291 (local address 127.15.232.190:41527)
0808 20:56:56.272542 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:56.272544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:56.272550 (+ 6us) server_negotiation.cc:407] Connection header received
0808 20:56:56.272569 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:56.272570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:56.272580 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:56.272634 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:56.272944 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.273639 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.274268 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.274450 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.274498 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.276848 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:41291: BlockingRecv error: recv got EOF from 127.15.232.131:41291 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 20:56:56.276937 29831 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:56 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:56.989730 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:56.984506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:39497)
0808 20:56:56.984664 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:56.984683 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:56.984742 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:56.984905 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:56.984906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:56.985079 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:56.985130 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.985131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.985809 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.985811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.986422 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:56.986424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.986779 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.986780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:56.986866 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.987484 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:56.987490 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:56.989643 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
W20260808 20:56:56.989799 29842 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:56.984621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:39497 (local address 127.15.232.190:41527)
0808 20:56:56.984763 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:56.984764 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:56.984774 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:56:56.984802 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:56.984803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:56.984812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:56.984884 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:56.985202 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.985710 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.986509 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:56.986704 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:56.986759 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:56.989744 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:39497: BlockingRecv error: recv got EOF from 127.15.232.129:39497 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
Aug 08 20:56:57 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:57 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:57.144313 29844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:57.138988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:57311)
0808 20:56:57.139198 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:57.139211 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:57.139303 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:57.139543 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:57.139544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:57.139734 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:57.139789 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:57.139791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.140475 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.140476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:57.141045 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:57.141047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.141394 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.141395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:57.141437 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:57.141947 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:57.141952 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:57.144238 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":119,"threads_started":1}
W20260808 20:56:57.144421 29842 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:57.139140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:57311 (local address 127.15.232.190:41527)
0808 20:56:57.139205 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:57.139212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:57.139339 (+ 127us) server_negotiation.cc:407] Connection header received
0808 20:56:57.139375 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:57.139376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:57.139390 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:57.139466 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:57.139885 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.140393 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.141133 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.141320 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.141375 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:57.144327 (+ 2952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:57311: BlockingRecv error: recv got EOF from 127.15.232.130:57311 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 20:56:57 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:57 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:57.282603 29845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:57.277333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:39603)
0808 20:56:57.277536 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:57.277546 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:57.277603 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:57.277774 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:57.277775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:57.277977 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:57.278027 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:57.278028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.278889 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.278890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:57.279560 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:57.279563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.279894 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.279895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:57.279947 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:57.280478 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:57.280483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:57.282474 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":117,"threads_started":1}
W20260808 20:56:57.282621 29842 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:57.277436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:39603 (local address 127.15.232.190:41527)
0808 20:56:57.277505 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:57.277506 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:57.277602 (+ 96us) server_negotiation.cc:407] Connection header received
0808 20:56:57.277646 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:57.277647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:57.277660 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:57.277721 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:57.278140 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.278792 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.279633 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.279817 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.279885 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:57.282571 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:39603: BlockingRecv error: recv got EOF from 127.15.232.131:39603 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 20:56:57 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:57 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:57.995195 29846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:57.990310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:59341)
0808 20:56:57.990504 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:57.990517 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:57.990591 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:57.990758 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:57.990760 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:57.990956 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:57.991008 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:57.991010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.991699 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.991700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:57.992233 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:57.992235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.992603 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.992604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:57.992646 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:57.993126 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:57.993131 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:57.995132 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":112,"threads_started":1}
W20260808 20:56:57.995406 29847 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:57.990408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:59341 (local address 127.15.232.190:41527)
0808 20:56:57.990525 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:57.990526 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:57.990600 (+ 74us) server_negotiation.cc:407] Connection header received
0808 20:56:57.990640 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:57.990641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:57.990651 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:57.990713 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:57.991084 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.991625 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.992319 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:57.992518 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:57.992593 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:57.995329 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:59341: BlockingRecv error: recv got EOF from 127.15.232.129:59341 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
Aug 08 20:56:58 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:58 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:58.149816 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:58.144897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:34441)
0808 20:56:58.145050 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:58.145061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:58.145164 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:58.145355 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:58.145356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:58.145628 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:58.145688 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:58.145691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.146355 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.146356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:58.146937 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:58.146939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.147333 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.147334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:58.147385 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:58.147908 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:58.147913 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:58.149756 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
W20260808 20:56:58.149991 29847 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:58.144957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:34441 (local address 127.15.232.190:41527)
0808 20:56:58.145012 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:58.145014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:58.145143 (+ 129us) server_negotiation.cc:407] Connection header received
0808 20:56:58.145251 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:58.145252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:58.145267 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:58.145327 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:58.145771 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.146271 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.147024 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.147236 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.147294 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:58.149901 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:34441: BlockingRecv error: recv got EOF from 127.15.232.130:34441 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 20:56:58 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:58 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:58.288601 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:58.283153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:44559)
0808 20:56:58.283304 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:58.283321 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:58.283402 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:58.283659 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:58.283660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:58.283858 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:58.283907 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:58.283909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.284649 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.284651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:58.285294 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:58.285297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.285720 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.285722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:58.285786 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:58.286311 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:58.286321 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:58.288511 (+ 2190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
W20260808 20:56:58.288702 29847 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:58.283341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:44559 (local address 127.15.232.190:41527)
0808 20:56:58.283409 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:58.283412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:58.283477 (+ 65us) server_negotiation.cc:407] Connection header received
0808 20:56:58.283514 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:58.283515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:58.283530 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:58.283645 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:58.283995 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.284547 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.285419 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.285620 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.285696 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:58.288620 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:44559: BlockingRecv error: recv got EOF from 127.15.232.131:44559 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 20:56:59 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:59 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:59.006038 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:58.995828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:54719)
0808 20:56:58.995995 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:58.996015 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:58.996078 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:58.996220 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:58.996220 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:58.996402 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:58.996454 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:58.996456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.997159 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.997160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:58.997736 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:58.997738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.998117 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.998118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:58.998174 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:59.003938 (+ 5764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:59.003945 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:59.005940 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":109,"threads_started":1}
W20260808 20:56:59.006150 29852 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:58.995886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:54719 (local address 127.15.232.190:41527)
0808 20:56:58.996028 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:58.996030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:58.996081 (+ 51us) server_negotiation.cc:407] Connection header received
0808 20:56:58.996123 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:58.996124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:58.996136 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:58.996200 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:58.996520 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.997051 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.997827 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:58.998025 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:58.998089 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:59.006063 (+ 7974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:54719: BlockingRecv error: recv got EOF from 127.15.232.129:54719 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
Aug 08 20:56:59 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:59 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:59.155586 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:59.150587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:55357)
0808 20:56:59.150862 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:59.150874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:59.150971 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:59.151138 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:59.151139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:59.151333 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:59.151415 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:59.151417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:59.152159 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:59.152160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:59.152693 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:59.152694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:59.153069 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:59.153070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:59.153127 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:59.153627 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:59.153631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:59.155487 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":92,"threads_started":1}
W20260808 20:56:59.155637 29852 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:59.150634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:55357 (local address 127.15.232.190:41527)
0808 20:56:59.150855 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:59.150857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:59.150952 (+ 95us) server_negotiation.cc:407] Connection header received
0808 20:56:59.151056 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:59.151057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:59.151071 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:59.151122 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:59.151537 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:59.152069 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:59.152783 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:59.152990 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:59.153070 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:59.155573 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:55357: BlockingRecv error: recv got EOF from 127.15.232.130:55357 (error 108)
Metrics: {"server-negotiator.queue_time_us":185}
Aug 08 20:56:59 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:56:59 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:56:59.294601 29855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:56:59.289241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:42161)
0808 20:56:59.289460 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 20:56:59.289474 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:56:59.289578 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:56:59.289775 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:56:59.289776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:56:59.289984 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:56:59.290034 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:59.290036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:59.290734 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:59.290735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:59.291267 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:56:59.291268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:56:59.291712 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:56:59.291713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:56:59.291779 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:59.292376 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:56:59.292382 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:56:59.294522 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":107,"threads_started":1}
W20260808 20:56:59.294872 29852 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:56:59.289364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:42161 (local address 127.15.232.190:41527)
0808 20:56:59.289466 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 20:56:59.289469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:56:59.289556 (+ 87us) server_negotiation.cc:407] Connection header received
0808 20:56:59.289664 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:56:59.289665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:56:59.289686 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:56:59.289764 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:56:59.290121 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:59.290638 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:59.291358 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:56:59.291622 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:56:59.291700 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:56:59.294657 (+ 2957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:42161: BlockingRecv error: recv got EOF from 127.15.232.131:42161 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 20:57:00 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:00 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:00.011457 29856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:00.006672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:49251)
0808 20:57:00.006871 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:00.006882 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:00.006940 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:00.007115 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:00.007115 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:00.007289 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:00.007341 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:00.007343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.008042 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.008043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:00.008603 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:00.008605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.008978 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.008979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:00.009020 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:00.009512 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:00.009517 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:00.011368 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":124,"threads_started":1}
W20260808 20:57:00.011677 29857 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:00.006833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:49251 (local address 127.15.232.190:41527)
0808 20:57:00.006988 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:00.006990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:00.006999 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:57:00.007024 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:00.007024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:00.007035 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:00.007096 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:00.007408 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.007953 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.008703 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.008903 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.008968 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:00.011589 (+ 2621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:49251: BlockingRecv error: recv got EOF from 127.15.232.129:49251 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
Aug 08 20:57:00 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:00 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:00.161754 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:00.156514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:36925)
0808 20:57:00.156678 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:00.156696 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:00.156758 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:00.156928 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:00.156929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:00.157102 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:00.157153 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:00.157154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.157796 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.157797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:00.158324 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:00.158326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.158963 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.158965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:00.159037 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:00.159573 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:00.159578 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:00.161657 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
W20260808 20:57:00.161815 29857 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:00.156588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:36925 (local address 127.15.232.190:41527)
0808 20:57:00.156690 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:00.156692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:00.156757 (+ 65us) server_negotiation.cc:407] Connection header received
0808 20:57:00.156830 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:00.156831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:00.156842 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:00.156906 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:00.157219 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.157714 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.158433 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.158857 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.158959 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:00.161751 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:36925: BlockingRecv error: recv got EOF from 127.15.232.130:36925 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 20:57:00 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:00 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:00.302173 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:00.295225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:34223)
0808 20:57:00.295517 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:00.295529 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:00.295690 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:00.296506 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:00.296507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:00.296767 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:00.296847 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:00.296849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.297654 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.297656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:00.298218 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:00.298220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.298650 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.298651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:00.298705 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:00.299297 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:00.299303 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:00.302108 (+ 2805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":114,"threads_started":1}
W20260808 20:57:00.302338 29857 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:00.295330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:34223 (local address 127.15.232.190:41527)
0808 20:57:00.295476 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:00.295479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:00.295610 (+ 131us) server_negotiation.cc:407] Connection header received
0808 20:57:00.296312 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:00.296313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:00.296330 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:00.296412 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:00.296979 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.297532 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.298333 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:00.298558 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:00.298625 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:00.302252 (+ 3627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:34223: BlockingRecv error: recv got EOF from 127.15.232.131:34223 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 08 20:57:01 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:01 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:01.017246 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:01.012115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:45403)
0808 20:57:01.012325 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:01.012338 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:01.012398 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:01.012643 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:01.012645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:01.012847 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:01.012899 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:01.012901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.013627 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.013628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:01.014162 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:01.014163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.014557 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.014559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:01.014627 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:01.015224 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:01.015232 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:01.017184 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":109,"threads_started":1}
W20260808 20:57:01.017421 29863 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:01.012271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:45403 (local address 127.15.232.190:41527)
0808 20:57:01.012401 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:01.012402 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:01.012444 (+ 42us) server_negotiation.cc:407] Connection header received
0808 20:57:01.012501 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:01.012502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:01.012513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:01.012570 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:01.012996 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.013542 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.014240 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.014433 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.014517 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:01.017333 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:45403: BlockingRecv error: recv got EOF from 127.15.232.129:45403 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
Aug 08 20:57:01 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:01 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:01.167590 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:01.162325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:34577)
0808 20:57:01.162469 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:01.162482 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:01.162547 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:01.162713 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:01.162714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:01.162913 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:01.162985 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:01.162987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.163754 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.163755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:01.164300 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:01.164302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.164689 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.164690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:01.164750 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:01.165311 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:01.165316 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:01.167526 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
W20260808 20:57:01.167732 29863 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:01.162452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:34577 (local address 127.15.232.190:41527)
0808 20:57:01.162521 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:01.162523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:01.162532 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:57:01.162608 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:01.162609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:01.162623 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:01.162699 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:01.163077 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.163659 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.164399 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.164608 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.164691 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:01.167646 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:34577: BlockingRecv error: recv got EOF from 127.15.232.130:34577 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 20:57:01 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:01 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:01.307804 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:01.302814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:33669)
0808 20:57:01.302967 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:01.302981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:01.303052 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:01.303184 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:01.303185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:01.303365 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:01.303416 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:01.303418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.304151 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.304152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:01.304724 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:01.304726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.305074 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.305075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:01.305126 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:01.305700 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:01.305706 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:01.307745 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
W20260808 20:57:01.307912 29863 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:01.302939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:33669 (local address 127.15.232.190:41527)
0808 20:57:01.302990 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:01.302992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:01.303086 (+ 94us) server_negotiation.cc:407] Connection header received
0808 20:57:01.303113 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:01.303114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:01.303127 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:01.303170 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:01.303520 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.304033 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.304808 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:01.304987 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:01.305041 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:01.307839 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:33669: BlockingRecv error: recv got EOF from 127.15.232.131:33669 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 20:57:02 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:02 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:02.023211 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:02.017828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:55765)
0808 20:57:02.018070 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:02.018081 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:02.018150 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:02.018319 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:02.018319 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:02.018499 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:02.018550 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:02.018552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.019251 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.019252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:02.019956 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:02.019958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.020439 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.020441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:02.020508 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:02.021067 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:02.021073 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:02.023104 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":134,"threads_started":1}
W20260808 20:57:02.023254 29867 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:02.017971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:55765 (local address 127.15.232.190:41527)
0808 20:57:02.018119 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:02.018121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:02.018141 (+ 20us) server_negotiation.cc:407] Connection header received
0808 20:57:02.018202 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:02.018203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:02.018215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:02.018280 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:02.018645 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.019161 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.020049 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.020259 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.020418 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:02.023183 (+ 2765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:55765: BlockingRecv error: recv got EOF from 127.15.232.129:55765 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
Aug 08 20:57:02 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:02 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:02.173094 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:02.168132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:47169)
0808 20:57:02.168292 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:02.168303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:02.168364 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:02.168570 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:02.168572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:02.168748 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:02.168800 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:02.168801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.169525 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.169526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:02.170076 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:02.170078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.170457 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.170458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:02.170500 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:02.170965 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:02.170970 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:02.173025 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
W20260808 20:57:02.173236 29867 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:02.168227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:47169 (local address 127.15.232.190:41527)
0808 20:57:02.168281 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:02.168282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:02.168348 (+ 66us) server_negotiation.cc:407] Connection header received
0808 20:57:02.168418 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:02.168419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:02.168429 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:02.168484 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:02.168890 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.169428 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.170167 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.170358 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.170426 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:02.173153 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:47169: BlockingRecv error: recv got EOF from 127.15.232.130:47169 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 08 20:57:02 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:02 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:02.313431 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:02.308388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:51127)
0808 20:57:02.308568 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:02.308587 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:02.308648 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:02.308803 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:02.308804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:02.308994 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:02.309044 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:02.309046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.309728 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.309729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:02.310283 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:02.310284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.310650 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.310650 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:02.310694 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:02.311267 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:02.311272 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:02.313331 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":127,"threads_started":1}
W20260808 20:57:02.313470 29867 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:02.308504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:51127 (local address 127.15.232.190:41527)
0808 20:57:02.308576 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:02.308578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:02.308634 (+ 56us) server_negotiation.cc:407] Connection header received
0808 20:57:02.308722 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:02.308723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:02.308735 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:02.308780 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:02.309129 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.309645 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.310363 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:02.310562 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:02.310618 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:02.313409 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:51127: BlockingRecv error: recv got EOF from 127.15.232.131:51127 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 20:57:03 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:03 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:03.028692 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:03.023809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:32787)
0808 20:57:03.023983 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:03.023996 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:03.024058 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:03.024205 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:03.024206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:03.024373 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:03.024453 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:03.024455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.025150 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.025151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:03.025911 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:03.025914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.026295 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.026299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:03.026341 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:03.026852 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:03.026862 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:03.028633 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":127,"threads_started":1}
W20260808 20:57:03.028838 29872 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:03.023913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:32787 (local address 127.15.232.190:41527)
0808 20:57:03.024052 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:03.024054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:03.024070 (+ 16us) server_negotiation.cc:407] Connection header received
0808 20:57:03.024116 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:03.024116 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:03.024127 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:03.024182 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:03.024526 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.025057 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.026008 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.026224 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.026282 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:03.028757 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:32787: BlockingRecv error: recv got EOF from 127.15.232.129:32787 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
Aug 08 20:57:03 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:03 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:03.179191 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:03.173670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:41063)
0808 20:57:03.173882 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:03.173895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:03.173995 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:03.174177 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:03.174178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:03.174367 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:03.174417 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:03.174419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.175196 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.175197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:03.175756 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:03.175758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.176139 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.176140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:03.176204 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:03.176820 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:03.176825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:03.179130 (+ 2305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":119,"threads_started":1}
W20260808 20:57:03.179333 29872 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:03.173799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:41063 (local address 127.15.232.190:41527)
0808 20:57:03.173886 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:03.173888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:03.173975 (+ 87us) server_negotiation.cc:407] Connection header received
0808 20:57:03.174075 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:03.174076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:03.174090 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:03.174168 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:03.174540 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.175095 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.175842 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.176047 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.176137 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:03.179249 (+ 3112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:41063: BlockingRecv error: recv got EOF from 127.15.232.130:41063 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 20:57:03 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:03 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:03.318715 29876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:03.314038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:45907)
0808 20:57:03.314179 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:03.314198 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:03.314249 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:03.314391 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:03.314391 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:03.314563 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:03.314612 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:03.314614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.315273 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.315274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:03.315874 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:03.315876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.316236 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.316237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:03.316282 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:03.316802 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:03.316808 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:03.318647 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
W20260808 20:57:03.318926 29872 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:03.314188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:45907 (local address 127.15.232.190:41527)
0808 20:57:03.314240 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:03.314246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:03.314257 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:57:03.314300 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:03.314301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:03.314315 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:03.314373 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:03.314694 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.315200 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.315971 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:03.316160 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:03.316225 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:03.318847 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:45907: BlockingRecv error: recv got EOF from 127.15.232.131:45907 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 08 20:57:04 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:04 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:04.039641 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:04.029359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:35239)
0808 20:57:04.029508 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:04.029519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:04.029571 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:04.029882 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:04.029884 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:04.030086 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:04.030136 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:04.030139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.030905 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.030906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:04.031584 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:04.031587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.032069 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.032070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:04.032126 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:04.037291 (+ 5165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:04.037297 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:04.039571 (+ 2274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
W20260808 20:57:04.039778 29878 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:04.029501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:35239 (local address 127.15.232.190:41527)
0808 20:57:04.029650 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:04.029652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:04.029660 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:57:04.029733 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:04.029734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:04.029745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:04.029818 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:04.030250 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.030786 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.031690 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.031961 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.032040 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:04.039685 (+ 7645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:35239: BlockingRecv error: recv got EOF from 127.15.232.129:35239 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
Aug 08 20:57:04 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:04 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:04.185096 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:04.179756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:54359)
0808 20:57:04.179975 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:04.179990 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:04.180059 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:04.180201 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:04.180202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:04.180397 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:04.180477 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:04.180479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.181335 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.181336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:04.181875 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:04.181877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.182295 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.182297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:04.182365 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:04.182924 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:04.182929 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:04.185022 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
W20260808 20:57:04.185227 29878 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:04.179900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:54359 (local address 127.15.232.190:41527)
0808 20:57:04.180001 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:04.180003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:04.180038 (+ 35us) server_negotiation.cc:407] Connection header received
0808 20:57:04.180127 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:04.180128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:04.180142 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:04.180194 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:04.180568 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.181233 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.181998 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.182203 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.182291 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:04.185124 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:54359: BlockingRecv error: recv got EOF from 127.15.232.130:54359 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 20:57:04 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:04 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:04.324321 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:04.319319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:54381)
0808 20:57:04.319501 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:04.319515 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:04.319594 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:04.319732 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:04.319732 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:04.319898 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:04.319949 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:04.319950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.320754 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.320755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:04.321309 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:04.321311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.321753 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.321754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:04.321804 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:04.322272 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:04.322277 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:04.324246 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":113,"threads_started":1}
W20260808 20:57:04.324389 29878 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:04.319404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:54381 (local address 127.15.232.190:41527)
0808 20:57:04.319540 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:04.319542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:04.319585 (+ 43us) server_negotiation.cc:407] Connection header received
0808 20:57:04.319659 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:04.319659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:04.319680 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:04.319720 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:04.320025 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.320668 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.321444 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:04.321646 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:04.321716 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:04.324338 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:54381: BlockingRecv error: recv got EOF from 127.15.232.131:54381 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 08 20:57:05 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:05 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:05.045046 29882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:05.040159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:34655)
0808 20:57:05.040302 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:05.040316 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:05.040396 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:05.040539 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:05.040540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:05.040715 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:05.040766 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:05.040768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.041416 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.041417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:05.041956 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:05.041957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.042434 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.042435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:05.042490 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:05.043002 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:05.043007 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:05.044977 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
W20260808 20:57:05.045219 29884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:05.040245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:34655 (local address 127.15.232.190:41527)
0808 20:57:05.040409 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:05.040411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:05.040421 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:57:05.040448 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:05.040448 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:05.040459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:05.040518 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:05.040840 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.041347 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.042061 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.042310 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.042398 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:05.045120 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:34655: BlockingRecv error: recv got EOF from 127.15.232.129:34655 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
Aug 08 20:57:05 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:05 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:05.191046 29885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:05.185574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:43199)
0808 20:57:05.185812 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:05.185825 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:05.185923 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:05.186094 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:05.186095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:05.186298 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:05.186349 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:05.186351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.187144 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.187145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:05.187738 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:05.187740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.188122 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.188123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:05.188186 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:05.188660 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:05.188665 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:05.190972 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":136,"threads_started":1}
W20260808 20:57:05.191111 29884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:05.185699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:43199 (local address 127.15.232.190:41527)
0808 20:57:05.185798 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:05.185801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:05.185904 (+ 103us) server_negotiation.cc:407] Connection header received
0808 20:57:05.185989 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:05.185990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:05.186004 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:05.186085 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:05.186436 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.187054 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.187829 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.188034 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.188121 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:05.191062 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:43199: BlockingRecv error: recv got EOF from 127.15.232.130:43199 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 20:57:05 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:05 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:05.330525 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:05.324934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:36043)
0808 20:57:05.325198 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:05.325213 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:05.325324 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:05.325568 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:05.325568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:05.325792 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:05.325846 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:05.325849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.326670 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.326672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:05.327267 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:05.327269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.327786 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.327787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:05.327862 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:05.328393 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:05.328398 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:05.330462 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":131,"threads_started":1}
W20260808 20:57:05.330668 29884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:05.325058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:36043 (local address 127.15.232.190:41527)
0808 20:57:05.325173 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:05.325175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:05.325310 (+ 135us) server_negotiation.cc:407] Connection header received
0808 20:57:05.325470 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:05.325472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:05.325490 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:05.325561 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:05.325948 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.326548 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.327395 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:05.327675 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:05.327783 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:05.330584 (+ 2801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:36043: BlockingRecv error: recv got EOF from 127.15.232.131:36043 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 20:57:06 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:06 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:06.050940 29892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:06.045600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:43485)
0808 20:57:06.045804 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:06.045815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:06.045876 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:06.046059 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:06.046060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:06.046233 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:06.046283 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:06.046285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.046981 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.046981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:06.047606 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:06.047609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.047980 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.047981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:06.048027 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:06.048693 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:06.048700 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:06.050855 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":117,"threads_started":1}
W20260808 20:57:06.051069 29893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:06.045719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:43485 (local address 127.15.232.190:41527)
0808 20:57:06.045858 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:06.045860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:06.045870 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:57:06.045963 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:06.045964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:06.045976 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:06.046045 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:06.046346 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.046899 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.047692 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.047899 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.047957 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:06.050973 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:43485: BlockingRecv error: recv got EOF from 127.15.232.129:43485 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
Aug 08 20:57:06 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:06 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:06.197412 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:06.191638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:51533)
0808 20:57:06.191815 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:06.191831 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:06.191955 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:06.192128 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:06.192130 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:06.192334 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:06.192386 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:06.192388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.193497 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.193499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:06.194111 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:06.194113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.194646 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.194647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:06.194712 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:06.195433 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:06.195438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:06.197352 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
W20260808 20:57:06.197616 29893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:06.191739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:51533 (local address 127.15.232.190:41527)
0808 20:57:06.191820 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:06.191823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:06.191929 (+ 106us) server_negotiation.cc:407] Connection header received
0808 20:57:06.192036 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:06.192037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:06.192050 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:06.192113 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:06.192486 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.193384 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.194203 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.194563 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.194648 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:06.197539 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:51533: BlockingRecv error: recv got EOF from 127.15.232.130:51533 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 20:57:06 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:06 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:06.336270 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:06.331116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:52051)
0808 20:57:06.331283 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:06.331299 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:06.331368 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:06.331548 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:06.331550 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:06.331730 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:06.331780 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:06.331782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.332507 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.332507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:06.333164 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:06.333167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.333499 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.333500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:06.333545 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:06.334131 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:06.334136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:06.336199 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
W20260808 20:57:06.336400 29893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:06.331273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:52051 (local address 127.15.232.190:41527)
0808 20:57:06.331342 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:06.331344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:06.331355 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:57:06.331447 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:06.331448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:06.331459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:06.331529 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:06.331876 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.332410 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.333241 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:06.333425 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:06.333490 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:06.336317 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:52051: BlockingRecv error: recv got EOF from 127.15.232.131:52051 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 20:57:07 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:07 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:07.058853 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:07.051711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:40775)
0808 20:57:07.051884 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:07.051897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:07.051973 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:07.052207 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:07.052208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:07.052390 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:07.052439 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:07.052441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.053112 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.053113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:07.053702 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:07.053704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.054136 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.054137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:07.054185 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:07.054945 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:07.054951 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:07.058738 (+ 3787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
W20260808 20:57:07.059051 29898 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:07.051908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:40775 (local address 127.15.232.190:41527)
0808 20:57:07.052043 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:07.052045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:07.052056 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:57:07.052085 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:07.052086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:07.052096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:07.052159 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:07.052515 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.053033 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.053847 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.054058 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.054123 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:07.058898 (+ 4775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:40775: BlockingRecv error: recv got EOF from 127.15.232.129:40775 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
Aug 08 20:57:07 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:07 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:07.202826 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:07.197973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:56309)
0808 20:57:07.198128 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:07.198138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:07.198218 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:07.198403 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:07.198404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:07.198672 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:07.198729 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:07.198731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.199389 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.199389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:07.200001 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:07.200003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.200360 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.200361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:07.200402 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:07.200875 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:07.200880 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:07.202753 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
W20260808 20:57:07.202972 29898 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:07.198125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:56309 (local address 127.15.232.190:41527)
0808 20:57:07.198193 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:07.198195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:07.198206 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:57:07.198303 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:07.198304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:07.198319 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:07.198382 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:07.198804 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.199317 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.200097 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.200290 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.200350 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:07.202885 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:56309: BlockingRecv error: recv got EOF from 127.15.232.130:56309 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 20:57:07 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:07 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:07.341840 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:07.336945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:49411)
0808 20:57:07.337124 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:07.337139 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:07.337225 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:07.337406 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:07.337407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:07.337618 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:07.337663 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:07.337665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.338361 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.338362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:07.338910 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:07.338912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.339283 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.339284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:07.339362 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:07.339950 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:07.339955 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:07.341769 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
W20260808 20:57:07.341988 29898 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:07.337006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:49411 (local address 127.15.232.190:41527)
0808 20:57:07.337096 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:07.337098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:07.337204 (+ 106us) server_negotiation.cc:407] Connection header received
0808 20:57:07.337307 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:07.337307 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:07.337330 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:07.337398 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:07.337758 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.338273 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.338999 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:07.339200 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:07.339282 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:07.341906 (+ 2624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:49411: BlockingRecv error: recv got EOF from 127.15.232.131:49411 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 20:57:08 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:08 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:08.064999 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:08.059612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:57393)
0808 20:57:08.059782 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:08.059794 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:08.059855 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:08.060029 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:08.060030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:08.060217 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:08.060269 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:08.060271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.061031 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.061032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:08.061617 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:08.061620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.062023 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.062023 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:08.062064 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:08.062768 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:08.062774 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:08.064937 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
W20260808 20:57:08.065143 29904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:08.059711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:57393 (local address 127.15.232.190:41527)
0808 20:57:08.059846 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:08.059862 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:08.059896 (+ 34us) server_negotiation.cc:407] Connection header received
0808 20:57:08.059923 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:08.059923 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:08.059935 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:08.059997 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:08.060361 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.060930 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.061726 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.061948 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.062014 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:08.065048 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:57393: BlockingRecv error: recv got EOF from 127.15.232.129:57393 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
Aug 08 20:57:08 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:08 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:08.208571 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:08.203401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:46115)
0808 20:57:08.203595 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:08.203606 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:08.203668 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:08.203851 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:08.203853 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:08.204056 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:08.204113 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:08.204114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.204842 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.204843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:08.205481 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:08.205483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.205856 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.205857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:08.205921 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:08.206398 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:08.206404 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:08.208510 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":129,"threads_started":1}
W20260808 20:57:08.208701 29904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:08.203531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:46115 (local address 127.15.232.190:41527)
0808 20:57:08.203611 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:08.203613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:08.203663 (+ 50us) server_negotiation.cc:407] Connection header received
0808 20:57:08.203723 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:08.203723 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:08.203734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:08.203794 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:08.204195 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.204754 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.205582 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.205769 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.205822 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:08.208621 (+ 2799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:46115: BlockingRecv error: recv got EOF from 127.15.232.130:46115 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 20:57:08 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:08 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:08.347266 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:08.342322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:43701)
0808 20:57:08.342497 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:08.342516 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:08.342612 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:08.342776 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:08.342777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:08.342985 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:08.343033 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:08.343035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.343824 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.343825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:08.344364 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:08.344366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.344732 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.344733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:08.344800 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:08.345296 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:08.345300 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:08.347205 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
W20260808 20:57:08.347399 29904 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:08.342430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:43701 (local address 127.15.232.190:41527)
0808 20:57:08.342501 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:08.342504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:08.342596 (+ 92us) server_negotiation.cc:407] Connection header received
0808 20:57:08.342671 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:08.342672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:08.342688 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:08.342766 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:08.343143 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.343730 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.344456 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:08.344653 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:08.344734 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:08.347313 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:43701: BlockingRecv error: recv got EOF from 127.15.232.131:43701 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 20:57:09 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:09 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:09.074987 29909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:09.065572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:56735)
0808 20:57:09.065773 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:09.065786 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:09.065846 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:09.066019 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:09.066019 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:09.066204 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:09.066255 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:09.066257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.066906 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.066907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:09.067576 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:09.067579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.067999 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.068000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:09.068053 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:09.072688 (+ 4635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:09.072694 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:09.074877 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":112,"threads_started":1}
W20260808 20:57:09.075057 29910 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:09.065680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:56735 (local address 127.15.232.190:41527)
0808 20:57:09.065815 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:09.065817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:09.065829 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:57:09.065920 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:09.065920 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:09.065932 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:09.065991 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:09.066323 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.066830 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.067694 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.067915 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.067986 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:09.074985 (+ 6999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:56735: BlockingRecv error: recv got EOF from 127.15.232.129:56735 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
Aug 08 20:57:09 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:09 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:09.214371 29911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:09.209170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:51211)
0808 20:57:09.209356 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:09.209367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:09.209455 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:09.209593 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:09.209594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:09.209770 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:09.209823 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:09.209824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.210619 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.210621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:09.211196 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:09.211198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.211689 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.211690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:09.211735 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:09.212263 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:09.212267 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:09.214256 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":83,"threads_started":1}
W20260808 20:57:09.214423 29910 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:09.209362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:51211 (local address 127.15.232.190:41527)
0808 20:57:09.209457 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:09.209459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:09.209495 (+ 36us) server_negotiation.cc:407] Connection header received
0808 20:57:09.209521 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:09.209522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:09.209533 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:09.209574 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:09.209924 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.210515 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.211295 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.211611 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.211673 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:09.214352 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:51211: BlockingRecv error: recv got EOF from 127.15.232.130:51211 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 20:57:09 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:09 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:09.353046 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:09.347949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:49741)
0808 20:57:09.348083 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:09.348096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:09.348161 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:09.348364 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:09.348365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:09.348545 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:09.348594 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:09.348596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.349311 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.349311 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:09.349977 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:09.349979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.350435 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.350436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:09.350508 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:09.351115 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:09.351121 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:09.352975 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
W20260808 20:57:09.353268 29910 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:09.348045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:49741 (local address 127.15.232.190:41527)
0808 20:57:09.348128 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:09.348130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:09.348144 (+ 14us) server_negotiation.cc:407] Connection header received
0808 20:57:09.348227 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:09.348228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:09.348242 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:09.348311 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:09.348697 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.349235 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.350069 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:09.350316 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:09.350385 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:09.353188 (+ 2803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:49741: BlockingRecv error: recv got EOF from 127.15.232.131:49741 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 20:57:10 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:10 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:10.081154 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:10.075648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:49195)
0808 20:57:10.075793 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:10.075807 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:10.075869 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:10.076042 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:10.076043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:10.076216 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:10.076265 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:10.076267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.076966 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.076967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:10.077635 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:10.077639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.078101 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.078103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:10.078153 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:10.078754 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:10.078760 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:10.081091 (+ 2331us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
W20260808 20:57:10.081350 29916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:10.075690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:49195 (local address 127.15.232.190:41527)
0808 20:57:10.075904 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:10.075906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:10.075915 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:57:10.075939 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:10.075939 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:10.075951 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:10.076017 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:10.076338 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.076854 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.077782 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.077995 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.078064 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:10.081250 (+ 3186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:49195: BlockingRecv error: recv got EOF from 127.15.232.129:49195 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":99,"threads_started":1}
Aug 08 20:57:10 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:10 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:10.219853 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:10.214992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:48031)
0808 20:57:10.215152 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:10.215163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:10.215235 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:10.215380 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:10.215381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:10.215582 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:10.215634 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:10.215636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.216341 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.216342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:10.216883 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:10.216885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.217240 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.217241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:10.217288 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:10.217763 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:10.217767 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:10.219692 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
W20260808 20:57:10.219889 29916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:10.215105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:48031 (local address 127.15.232.190:41527)
0808 20:57:10.215163 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:10.215165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:10.215218 (+ 53us) server_negotiation.cc:407] Connection header received
0808 20:57:10.215299 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:10.215300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:10.215311 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:10.215355 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:10.215734 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.216238 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.216980 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.217169 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.217222 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:10.219842 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:48031: BlockingRecv error: recv got EOF from 127.15.232.130:48031 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 20:57:10 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:10 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.131@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:10.358860 29921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:10.353680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.131:43273)
0808 20:57:10.353965 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:10.353977 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:10.354082 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:10.354306 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:10.354307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:10.354610 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:10.354675 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:10.354678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.355402 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.355403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:10.356009 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:10.356011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.356352 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.356352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:10.356394 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:10.356876 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:10.356881 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:10.358758 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":98,"threads_started":1}
W20260808 20:57:10.358915 29916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:10.353902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:43273 (local address 127.15.232.190:41527)
0808 20:57:10.353986 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:10.353988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:10.354080 (+ 92us) server_negotiation.cc:407] Connection header received
0808 20:57:10.354170 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:10.354171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:10.354190 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:10.354291 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:10.354770 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.355319 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.356094 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:10.356282 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:10.356341 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:10.358843 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.131:43273: BlockingRecv error: recv got EOF from 127.15.232.131:43273 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 20:57:11 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:11 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.129@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:11.086755 29923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:11.081705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.129:51323)
0808 20:57:11.081909 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:11.081945 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:11.082014 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:11.082191 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:11.082192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:11.082379 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:11.082449 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.082451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.083164 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.083165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.083750 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.083751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.084105 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.084106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.084149 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.084764 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:11.084769 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:11.086638 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":64,"threads_started":1}
W20260808 20:57:11.086804 29924 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:11.081840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:51323 (local address 127.15.232.190:41527)
0808 20:57:11.082012 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:11.082014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:11.082024 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:57:11.082084 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:11.082085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:11.082096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:11.082164 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:11.082555 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.083087 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.083834 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.084033 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.084092 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.086730 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.129:51323: BlockingRecv error: recv got EOF from 127.15.232.129:51323 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
Aug 08 20:57:11 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 20:57:11 dist-test-slave-c12x krb5kdc[28387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.232.130@KRBTEST.COM for oryx/127.15.232.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 20:57:11.225709 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:11.220619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41527 (local address 127.15.232.130:58631)
0808 20:57:11.220777 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:11.220789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:11.220873 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:11.221016 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:11.221017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:11.221192 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:11.221242 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.221244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.221927 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.221928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.222462 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.222463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.222855 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.222856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.222902 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.223582 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:11.223587 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:11.225648 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.232.190:41527: Server oryx/127.15.232.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
W20260808 20:57:11.225869 29924 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:11.220756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:58631 (local address 127.15.232.190:41527)
0808 20:57:11.220822 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:11.220825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:11.220857 (+ 32us) server_negotiation.cc:407] Connection header received
0808 20:57:11.220925 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:11.220926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:11.220937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:11.220996 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:11.221308 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.221830 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.222561 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.222762 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.222841 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.225799 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.232.130:58631: BlockingRecv error: recv got EOF from 127.15.232.130:58631 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 20:57:11.268158 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 29226
I20260808 20:57:11.281980 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 29442
I20260808 20:57:11.290856 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 29640
I20260808 20:57:11.299567 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 29120
2026-08-08T20:57:11Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17804 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T20:57:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:57:11Z Disabled control of system clock
I20260808 20:57:11.334091 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33447
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:45315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:33447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 20:57:11.420796 29931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:11.421028 29931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:11.421053 29931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:11.422220 29931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:57:11.422260 29931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:11.422273 29931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:11.422286 29931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:57:11.422298 29931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:57:11.423657 29931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:33447
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:11.423870 29931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:11.424134 29931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:11.429585 29945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:11.429600 29942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:11.429601 29940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:11.429729 29931 server_base.cc:1034] running on GCE node
I20260808 20:57:11.430258 29931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:11.430625 29931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:11.431795 29931 hybrid_clock.cc:648] HybridClock initialized: now 1786222631431748 us; error 63 us; skew 500 ppm
I20260808 20:57:11.433312 29931 webserver.cc:460] Webserver started at http://127.15.232.190:43563/ using document root <none> and password file <none>
I20260808 20:57:11.433602 29931 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:11.433678 29931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:11.433840 29931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:11.435027 29931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "3a9f17588a95ef36d05f5db7e0e0d2e2"
server_key_iv: "fb50a195a3c389567b43f11f14715fb5"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.435488 29931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "3a9f17588a95ef36d05f5db7e0e0d2e2"
server_key_iv: "fb50a195a3c389567b43f11f14715fb5"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.437171 29931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:11.438150 29953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.438341 29931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:11.438501 29931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "3a9f17588a95ef36d05f5db7e0e0d2e2"
server_key_iv: "fb50a195a3c389567b43f11f14715fb5"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.438598 29931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:11.452446 29931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:11.452813 29931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:11.452981 29931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:11.457471 29931 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:33447
I20260808 20:57:11.457512 30029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:33447 every 8 connection(s)
I20260808 20:57:11.457912 29931 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:57:11.458671 30031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:57:11.459489 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 29931
I20260808 20:57:11.459621 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:57:11.459815 16290 external_mini_cluster.cc:1428] Setting key 10b53d72a0bfc51cfa75779dcacaf8c8
I20260808 20:57:11.462055 30031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1: Bootstrap starting.
I20260808 20:57:11.462883 30031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1: Neither blocks nor log segments found. Creating new log.
I20260808 20:57:11.463213 30031 log.cc:824] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1: Log is configured to *not* fsync() on all Append() calls
I20260808 20:57:11.464067 30031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1: No bootstrap required, opened a new log
I20260808 20:57:11.465677 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:11.460425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59884 (local address 127.15.232.190:33447)
0808 20:57:11.460640 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:11.460647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:11.460673 (+ 26us) server_negotiation.cc:407] Connection header received
0808 20:57:11.461098 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:11.461105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:11.461167 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:11.461291 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:11.461546 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.462487 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.463697 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.464153 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.464294 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.464985 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:11.464988 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:11.464989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:57:11.465163 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:11.465195 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:57:11.465207 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:11.465259 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:11.465371 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 20:57:11.465458 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260808 20:57:11.466472 30031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33447 } }
I20260808 20:57:11.466643 30031 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:57:11.466694 30031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd1b3caf37f4b3db6185ba1e74efcd1, State: Initialized, Role: FOLLOWER
I20260808 20:57:11.466825 30031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33447 } }
I20260808 20:57:11.466924 30031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:57:11.466969 30031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:57:11.467018 30031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:57:11.467670 30031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33447 } }
I20260808 20:57:11.467796 30031 leader_election.cc:302] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bd1b3caf37f4b3db6185ba1e74efcd1; no voters:
I20260808 20:57:11.467979 30031 leader_election.cc:288] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:57:11.468084 30038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:57:11.468329 30038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [term 1 LEADER]: Becoming Leader. State: Replica: 0bd1b3caf37f4b3db6185ba1e74efcd1, State: Running, Role: LEADER
I20260808 20:57:11.468395 30031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:57:11.468492 30038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33447 } }
I20260808 20:57:11.468945 30039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33447 } } }
I20260808 20:57:11.468976 30041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd1b3caf37f4b3db6185ba1e74efcd1. Latest consensus state: current_term: 1 leader_uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd1b3caf37f4b3db6185ba1e74efcd1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33447 } } }
I20260808 20:57:11.469122 30039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [sys.catalog]: This master's current role is: LEADER
I20260808 20:57:11.469159 30041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1 [sys.catalog]: This master's current role is: LEADER
I20260808 20:57:11.469348 30056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:57:11.469985 30056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:57:11.471624 30056 catalog_manager.cc:1347] Generated new cluster ID: a79bff1cae0c4492b402ea6e6c924067
I20260808 20:57:11.471684 30056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:57:11.479280 30056 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:57:11.480079 30056 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:57:11.490706 30056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bd1b3caf37f4b3db6185ba1e74efcd1: Generated new TSK 0
I20260808 20:57:11.490935 30056 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 20:57:11.499980 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33447
--builtin_ntp_servers=127.15.232.148:45315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 20:57:11.586129 30069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:11.586386 30069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:11.586436 30069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:11.587723 30069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:11.587782 30069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:11.587826 30069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:57:11.589267 30069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:11.589560 30069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:11.589888 30069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:11.595244 30082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:11.595253 30077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:11.595257 30084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:11.595548 30069 server_base.cc:1034] running on GCE node
I20260808 20:57:11.595750 30069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:11.596025 30069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:11.597208 30069 hybrid_clock.cc:648] HybridClock initialized: now 1786222631597190 us; error 36 us; skew 500 ppm
I20260808 20:57:11.598608 30069 webserver.cc:460] Webserver started at http://127.15.232.129:36233/ using document root <none> and password file <none>
I20260808 20:57:11.598869 30069 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:11.598964 30069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:11.599118 30069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:11.600296 30069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "7959c53afd8545d888a13a858e2db44f"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "7c1047828b562541cd86d2e052c23ad8"
server_key_iv: "e93050532aa4df6ce4efbbd67b37b6ab"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.600684 30069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "7959c53afd8545d888a13a858e2db44f"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "7c1047828b562541cd86d2e052c23ad8"
server_key_iv: "e93050532aa4df6ce4efbbd67b37b6ab"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.602078 30069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 20:57:11.602873 30092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.603127 30069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:57:11.603240 30069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "7959c53afd8545d888a13a858e2db44f"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "7c1047828b562541cd86d2e052c23ad8"
server_key_iv: "e93050532aa4df6ce4efbbd67b37b6ab"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.603322 30069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:11.640019 30069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:11.640348 30069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:11.640460 30069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:11.640686 30069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:57:11.641003 30069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:57:11.641034 30069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.641058 30069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:57:11.641080 30069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.647286 30069 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:43555
I20260808 20:57:11.647377 30263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:43555 every 8 connection(s)
I20260808 20:57:11.647770 30069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:57:11.653367 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:11.648621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:42997 (local address 127.15.232.190:33447)
0808 20:57:11.648682 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:11.648684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:11.648913 (+ 229us) server_negotiation.cc:407] Connection header received
0808 20:57:11.649445 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:11.649446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:11.649455 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:11.649518 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:11.650194 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.650705 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.651866 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.652077 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.652139 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.653056 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:11.653058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:11.653058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:57:11.653063 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:11.653078 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:57:11.653082 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:11.653129 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:11.653265 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 20:57:11.653299 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 20:57:11.653470 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:11.648582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33447 (local address 127.15.232.129:42997)
0808 20:57:11.648783 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:11.648827 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:11.649359 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:11.649594 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:11.649597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:11.649605 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:11.650118 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.650122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.650845 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.650846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.651765 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.651768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.652171 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.652172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.652293 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.652897 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:57:11.652897 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:11.652927 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:57:11.652951 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:57:11.653145 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:57:11.653150 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:57:11.653152 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 20:57:11.653207 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 20:57:11.653300 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260808 20:57:11.654189 30264 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33447
I20260808 20:57:11.654287 30264 heartbeater.cc:464] Registering TS with master...
I20260808 20:57:11.654547 30264 heartbeater.cc:511] Master 127.15.232.190:33447 requested a full tablet report, sending...
I20260808 20:57:11.655159 29973 ts_manager.cc:194] Registered new tserver with Master: 7959c53afd8545d888a13a858e2db44f (127.15.232.129:43555)
I20260808 20:57:11.655869 29973 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.129:42997
I20260808 20:57:11.656731 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 30069
I20260808 20:57:11.656854 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:57:11.657064 16290 external_mini_cluster.cc:1428] Setting key 563a6da8a17c0f6be7acf8ca78e810f2
I20260808 20:57:11.658061 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33447
--builtin_ntp_servers=127.15.232.148:45315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 20:57:11.749552 30268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:11.749787 30268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:11.749810 30268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:11.751040 30268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:11.751085 30268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:11.751116 30268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:57:11.752558 30268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:11.752847 30268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:11.753193 30268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:11.758689 30285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:11.758701 30279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:11.758728 30278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:11.759104 30268 server_base.cc:1034] running on GCE node
I20260808 20:57:11.759307 30268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:11.759605 30268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:11.760756 30268 hybrid_clock.cc:648] HybridClock initialized: now 1786222631760739 us; error 33 us; skew 500 ppm
I20260808 20:57:11.762050 30268 webserver.cc:460] Webserver started at http://127.15.232.130:33035/ using document root <none> and password file <none>
I20260808 20:57:11.762305 30268 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:11.762351 30268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:11.762552 30268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:11.763760 30268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "1e5e47c3e26f45fd97a4059324c2b8f8"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "bf408f647d9de2cd73013695f87146e6"
server_key_iv: "7e2a44188a3a5dba7813b6728ab0e2f6"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.764132 30268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "1e5e47c3e26f45fd97a4059324c2b8f8"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "bf408f647d9de2cd73013695f87146e6"
server_key_iv: "7e2a44188a3a5dba7813b6728ab0e2f6"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.765759 30268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:57:11.766752 30292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.766975 30268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:11.767114 30268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "1e5e47c3e26f45fd97a4059324c2b8f8"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "bf408f647d9de2cd73013695f87146e6"
server_key_iv: "7e2a44188a3a5dba7813b6728ab0e2f6"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.767200 30268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:11.785161 30268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:11.785521 30268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:11.785682 30268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:11.785956 30268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:57:11.786330 30268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:57:11.786388 30268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.786485 30268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:57:11.786518 30268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.793006 30268 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:34545
I20260808 20:57:11.793051 30450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:34545 every 8 connection(s)
I20260808 20:57:11.793440 30268 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:57:11.795362 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 30268
I20260808 20:57:11.795500 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:57:11.795917 16290 external_mini_cluster.cc:1428] Setting key 956aa54e57b7c8e7592b1cbfd25b6ccc
I20260808 20:57:11.797498 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33447
--builtin_ntp_servers=127.15.232.148:45315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 20:57:11.800092 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:11.794294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:37021 (local address 127.15.232.190:33447)
0808 20:57:11.794367 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:11.794370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:11.794667 (+ 297us) server_negotiation.cc:407] Connection header received
0808 20:57:11.795207 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:11.795208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:11.795220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:11.795321 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:11.796318 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.797069 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.798238 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.798584 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.798670 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.799785 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:11.799787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:11.799787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:57:11.799792 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:11.799808 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:57:11.799813 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:11.799866 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:11.799992 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 20:57:11.800025 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 20:57:11.800379 30456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:11.794287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33447 (local address 127.15.232.130:37021)
0808 20:57:11.794529 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:11.794566 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:11.795057 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:11.795423 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:11.795427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:11.795477 (+ 50us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:11.796086 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.796093 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.797207 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.797208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.798135 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.798138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.798687 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.798695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.798870 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.799594 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:57:11.799595 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:11.799640 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:57:11.799675 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:57:11.799883 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:57:11.799890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:57:11.799893 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:57:11.799985 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 20:57:11.800115 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":129,"threads_started":1}
I20260808 20:57:11.801011 30452 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33447
I20260808 20:57:11.801129 30452 heartbeater.cc:464] Registering TS with master...
I20260808 20:57:11.801337 30452 heartbeater.cc:511] Master 127.15.232.190:33447 requested a full tablet report, sending...
I20260808 20:57:11.801784 29973 ts_manager.cc:194] Registered new tserver with Master: 1e5e47c3e26f45fd97a4059324c2b8f8 (127.15.232.130:34545)
I20260808 20:57:11.802188 29973 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.130:37021
W20260808 20:57:11.886850 30458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:11.887115 30458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:11.887156 30458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:11.888378 30458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:11.888453 30458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:11.888499 30458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:57:11.889964 30458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:45315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:11.890280 30458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:11.890630 30458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:11.896013 30467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:11.896034 30470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:11.896013 30468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:11.896564 30458 server_base.cc:1034] running on GCE node
I20260808 20:57:11.896808 30458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:11.897017 30458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:11.898173 30458 hybrid_clock.cc:648] HybridClock initialized: now 1786222631898199 us; error 70 us; skew 500 ppm
I20260808 20:57:11.899605 30458 webserver.cc:460] Webserver started at http://127.15.232.131:40247/ using document root <none> and password file <none>
I20260808 20:57:11.899976 30458 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:11.900038 30458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:11.900147 30458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:11.901255 30458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "8ad450ff174c4d67a29655d8cf025653"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "9ef19af590747ad32ec4a99e5e5299df"
server_key_iv: "e69a8a8a78bb08f65f697cc35ebd6a19"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.901943 30458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "8ad450ff174c4d67a29655d8cf025653"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "9ef19af590747ad32ec4a99e5e5299df"
server_key_iv: "e69a8a8a78bb08f65f697cc35ebd6a19"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.903621 30458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 20:57:11.904459 30481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.904940 30458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:11.905088 30458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "8ad450ff174c4d67a29655d8cf025653"
format_stamp: "Formatted at 2026-08-08 20:57:11 on dist-test-slave-c12x"
server_key: "9ef19af590747ad32ec4a99e5e5299df"
server_key_iv: "e69a8a8a78bb08f65f697cc35ebd6a19"
server_key_version: "encryptionkey@0"
I20260808 20:57:11.905182 30458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:11.950605 30458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:11.951076 30458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:11.951241 30458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:11.951519 30458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:57:11.951891 30458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:57:11.951946 30458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.951989 30458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:57:11.952033 30458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:11.958325 30458 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:38739
I20260808 20:57:11.958376 30640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:38739 every 8 connection(s)
I20260808 20:57:11.958722 30458 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:57:11.964342 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:11.959550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:38455 (local address 127.15.232.190:33447)
0808 20:57:11.959622 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:11.959624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:11.959889 (+ 265us) server_negotiation.cc:407] Connection header received
0808 20:57:11.960369 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:11.960370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:11.960387 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:11.960464 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:11.961142 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.961737 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.962736 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.962965 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.963039 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.964034 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:11.964035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:11.964035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:57:11.964042 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:11.964058 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:57:11.964062 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:11.964122 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:11.964245 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 20:57:11.964272 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 20:57:11.964535 30643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:11.959542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33447 (local address 127.15.232.131:38455)
0808 20:57:11.959750 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:11.959785 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:11.960264 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:11.960515 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:11.960518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:11.960526 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:11.961040 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.961044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.961830 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.961831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.962615 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:11.962617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:11.963074 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:11.963075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:11.963206 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:11.963847 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:57:11.963848 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:11.963881 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:57:11.963908 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:57:11.964159 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:57:11.964164 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:57:11.964167 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:57:11.964228 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 20:57:11.964319 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":115,"threads_started":1}
I20260808 20:57:11.965147 30641 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33447
I20260808 20:57:11.965279 30641 heartbeater.cc:464] Registering TS with master...
I20260808 20:57:11.965576 30641 heartbeater.cc:511] Master 127.15.232.190:33447 requested a full tablet report, sending...
I20260808 20:57:11.965749 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 30458
I20260808 20:57:11.965868 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:57:11.966032 29973 ts_manager.cc:194] Registered new tserver with Master: 8ad450ff174c4d67a29655d8cf025653 (127.15.232.131:38739)
I20260808 20:57:11.966097 16290 external_mini_cluster.cc:1428] Setting key b4dbb0dfba5e50f904ee83b47478b3f5
I20260808 20:57:11.966483 29973 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.131:38455
I20260808 20:57:11.967149 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 20:57:11.969362 30034 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:11.968721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59900 (local address 127.15.232.190:33447)
0808 20:57:11.968766 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:11.968768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:11.968913 (+ 145us) server_negotiation.cc:407] Connection header received
0808 20:57:11.969025 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:11.969026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:11.969040 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:11.969123 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:11.969281 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59900: BlockingRecv error: recv got EOF from 127.0.0.1:59900 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 20:57:11.969494 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 30069
I20260808 20:57:11.993391 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 30268
I20260808 20:57:12.001854 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 30458
I20260808 20:57:12.013489 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 29931
2026-08-08T20:57:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (716 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T20:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:57:12Z Disabled control of system clock
I20260808 20:57:12.052122 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36197
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:36197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 20:57:12.138394 30656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:12.138622 30656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:12.138644 30656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:12.139750 30656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:57:12.139787 30656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:12.139801 30656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:12.139812 30656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:57:12.139824 30656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:57:12.141124 30656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:36197
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:12.141324 30656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:12.141574 30656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:12.147127 30669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:12.147202 30665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:12.147326 30656 server_base.cc:1034] running on GCE node
W20260808 20:57:12.147135 30664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:12.147691 30656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:12.147948 30656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:12.149142 30656 hybrid_clock.cc:648] HybridClock initialized: now 1786222632149130 us; error 32 us; skew 500 ppm
I20260808 20:57:12.150363 30656 webserver.cc:460] Webserver started at http://127.15.232.190:34093/ using document root <none> and password file <none>
I20260808 20:57:12.150563 30656 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:12.150628 30656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:12.150734 30656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:12.151887 30656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "550450946f42427b908d073407f82fd4"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "5e05f6cdaa7d017d59a270609f0298bf"
server_key_iv: "5782f76145b76fb4695f72a8106fc488"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.152258 30656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "550450946f42427b908d073407f82fd4"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "5e05f6cdaa7d017d59a270609f0298bf"
server_key_iv: "5782f76145b76fb4695f72a8106fc488"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.153628 30656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:57:12.154347 30675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.154657 30656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:12.154747 30656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "550450946f42427b908d073407f82fd4"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "5e05f6cdaa7d017d59a270609f0298bf"
server_key_iv: "5782f76145b76fb4695f72a8106fc488"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.154845 30656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:12.158669 30656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:12.158968 30656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:12.159106 30656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:12.163421 30656 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:36197
I20260808 20:57:12.163594 30743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:36197 every 8 connection(s)
I20260808 20:57:12.163918 30656 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:57:12.164673 30745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:57:12.167143 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 30656
I20260808 20:57:12.167227 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:57:12.167517 16290 external_mini_cluster.cc:1428] Setting key 742fdce780572b5773885a4ab528b295
I20260808 20:57:12.167891 30745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4: Bootstrap starting.
I20260808 20:57:12.168743 30745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4: Neither blocks nor log segments found. Creating new log.
I20260808 20:57:12.169118 30745 log.cc:824] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4: Log is configured to *not* fsync() on all Append() calls
I20260808 20:57:12.169971 30745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4: No bootstrap required, opened a new log
I20260808 20:57:12.170686 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:12.168144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43796 (local address 127.15.232.190:36197)
0808 20:57:12.168384 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:12.168393 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:12.168458 (+ 65us) server_negotiation.cc:407] Connection header received
0808 20:57:12.169133 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:12.169143 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:12.169200 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:12.169367 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:12.170190 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:12.170194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:12.170195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:57:12.170199 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:12.170247 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:57:12.170261 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:12.170367 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:12.170396 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 20:57:12.170456 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":98,"threads_started":1}
I20260808 20:57:12.172314 30745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550450946f42427b908d073407f82fd4" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36197 } }
I20260808 20:57:12.172451 30745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:57:12.172472 30745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550450946f42427b908d073407f82fd4, State: Initialized, Role: FOLLOWER
I20260808 20:57:12.172613 30745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550450946f42427b908d073407f82fd4" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36197 } }
I20260808 20:57:12.172716 30745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:57:12.172760 30745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:57:12.172829 30745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:57:12.173394 30745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550450946f42427b908d073407f82fd4" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36197 } }
I20260808 20:57:12.173519 30745 leader_election.cc:302] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 550450946f42427b908d073407f82fd4; no voters:
I20260808 20:57:12.173687 30745 leader_election.cc:288] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:57:12.173727 30751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:57:12.173913 30751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [term 1 LEADER]: Becoming Leader. State: Replica: 550450946f42427b908d073407f82fd4, State: Running, Role: LEADER
I20260808 20:57:12.173955 30745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:57:12.174049 30751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550450946f42427b908d073407f82fd4" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36197 } }
I20260808 20:57:12.174503 30756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 550450946f42427b908d073407f82fd4. Latest consensus state: current_term: 1 leader_uuid: "550450946f42427b908d073407f82fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550450946f42427b908d073407f82fd4" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36197 } } }
I20260808 20:57:12.174640 30756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [sys.catalog]: This master's current role is: LEADER
I20260808 20:57:12.174862 30765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:57:12.175051 30754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "550450946f42427b908d073407f82fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550450946f42427b908d073407f82fd4" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36197 } } }
I20260808 20:57:12.175135 30754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4 [sys.catalog]: This master's current role is: LEADER
I20260808 20:57:12.175508 30765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:57:12.177150 30765 catalog_manager.cc:1347] Generated new cluster ID: ac3a5f9912c947d4bbd41622149b7e8f
I20260808 20:57:12.177205 30765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:57:12.192889 30765 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:57:12.193109 30765 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:57:12.200789 30765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 550450946f42427b908d073407f82fd4: Generated new TSK 0
I20260808 20:57:12.201021 30765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 20:57:12.207038 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36197
--builtin_ntp_servers=127.15.232.148:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 20:57:12.292200 30781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:12.292451 30781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:12.292493 30781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:12.293736 30781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:12.293812 30781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:12.293861 30781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:57:12.295313 30781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:12.295619 30781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:12.295928 30781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:12.301090 30787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:12.301139 30786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:12.301326 30781 server_base.cc:1034] running on GCE node
W20260808 20:57:12.301507 30789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:12.301796 30781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:12.302129 30781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:12.303313 30781 hybrid_clock.cc:648] HybridClock initialized: now 1786222632303294 us; error 38 us; skew 500 ppm
I20260808 20:57:12.304853 30781 webserver.cc:460] Webserver started at http://127.15.232.129:41383/ using document root <none> and password file <none>
I20260808 20:57:12.305127 30781 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:12.305186 30781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:12.305325 30781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:12.306648 30781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "fb7fc5fc171c42a085eb4828d8a1d921"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "56db1fd79775388614252ff0a5c518b4"
server_key_iv: "9d254d9d9f4242fa1d1e427eee70d77d"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.306965 30781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "fb7fc5fc171c42a085eb4828d8a1d921"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "56db1fd79775388614252ff0a5c518b4"
server_key_iv: "9d254d9d9f4242fa1d1e427eee70d77d"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.308805 30781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260808 20:57:12.309609 30798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.309835 30781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:12.309952 30781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "fb7fc5fc171c42a085eb4828d8a1d921"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "56db1fd79775388614252ff0a5c518b4"
server_key_iv: "9d254d9d9f4242fa1d1e427eee70d77d"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.310048 30781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:12.313630 30781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:12.313922 30781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:12.314081 30781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:12.314309 30781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:57:12.314639 30781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:57:12.314697 30781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.314729 30781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:57:12.314760 30781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.320803 30781 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:35851
I20260808 20:57:12.320842 30952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:35851 every 8 connection(s)
I20260808 20:57:12.321193 30781 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:57:12.323148 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 30781
I20260808 20:57:12.323235 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:57:12.323472 16290 external_mini_cluster.cc:1428] Setting key 7cf135fdbd5f12ac3e0f05da8fef329e
I20260808 20:57:12.324385 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:12.321990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:43019 (local address 127.15.232.190:36197)
0808 20:57:12.322047 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:12.322050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:12.322266 (+ 216us) server_negotiation.cc:407] Connection header received
0808 20:57:12.322718 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:12.322719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:12.322724 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:12.322774 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:12.324103 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:12.324105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:12.324105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:57:12.324110 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:12.324126 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:57:12.324131 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:12.324202 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:12.324297 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 20:57:12.324313 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 20:57:12.324584 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36197
--builtin_ntp_servers=127.15.232.148:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 20:57:12.324604 30957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:12.321947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36197 (local address 127.15.232.129:43019)
0808 20:57:12.322139 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:12.322172 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:12.322632 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:12.322825 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:12.322828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:12.322836 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:12.323944 (+ 1108us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:57:12.323944 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:12.323973 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:57:12.324001 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:57:12.324202 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:57:12.324209 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:57:12.324212 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:57:12.324307 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 20:57:12.324372 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260808 20:57:12.325145 30954 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36197
I20260808 20:57:12.325248 30954 heartbeater.cc:464] Registering TS with master...
I20260808 20:57:12.325395 30954 heartbeater.cc:511] Master 127.15.232.190:36197 requested a full tablet report, sending...
I20260808 20:57:12.325928 30697 ts_manager.cc:194] Registered new tserver with Master: fb7fc5fc171c42a085eb4828d8a1d921 (127.15.232.129:35851)
W20260808 20:57:12.409972 30959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:12.410202 30959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:12.410225 30959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:12.411463 30959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:12.411504 30959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:12.411530 30959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:57:12.413034 30959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:12.413298 30959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:12.413645 30959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:12.418880 30964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:12.418859 30969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:12.418861 30965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:12.419137 30959 server_base.cc:1034] running on GCE node
I20260808 20:57:12.419345 30959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:12.419608 30959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:12.420789 30959 hybrid_clock.cc:648] HybridClock initialized: now 1786222632420777 us; error 29 us; skew 500 ppm
I20260808 20:57:12.422235 30959 webserver.cc:460] Webserver started at http://127.15.232.130:33897/ using document root <none> and password file <none>
I20260808 20:57:12.422436 30959 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:12.422479 30959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:12.422576 30959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:12.423786 30959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "63387e83b8cc42f180cac1c08ee15fcc"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "42327f765ab5f0792683e28ea1c2a2d4"
server_key_iv: "d92ee95c6d820ae3f80feb9763c36fa2"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.424122 30959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "63387e83b8cc42f180cac1c08ee15fcc"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "42327f765ab5f0792683e28ea1c2a2d4"
server_key_iv: "d92ee95c6d820ae3f80feb9763c36fa2"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.425625 30959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:57:12.426462 30979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.426707 30959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:12.426808 30959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "63387e83b8cc42f180cac1c08ee15fcc"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "42327f765ab5f0792683e28ea1c2a2d4"
server_key_iv: "d92ee95c6d820ae3f80feb9763c36fa2"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.426882 30959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:12.430532 30959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:12.430771 30959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:12.430882 30959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:12.431058 30959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:57:12.431324 30959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:57:12.431367 30959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.431391 30959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:57:12.431406 30959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.437551 30959 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:42493
I20260808 20:57:12.437628 31148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:42493 every 8 connection(s)
I20260808 20:57:12.438001 30959 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:57:12.440653 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 30959
I20260808 20:57:12.440750 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:57:12.440973 16290 external_mini_cluster.cc:1428] Setting key 6818555c709fda530ca9c8a48be888fe
I20260808 20:57:12.441461 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:12.438855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:54599 (local address 127.15.232.190:36197)
0808 20:57:12.438957 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:12.438959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:12.439162 (+ 203us) server_negotiation.cc:407] Connection header received
0808 20:57:12.439723 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:12.439725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:12.439732 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:12.439805 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:12.441122 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:12.441123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:12.441123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:57:12.441129 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:12.441147 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:57:12.441152 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:12.441226 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:12.441345 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 20:57:12.441398 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 20:57:12.441646 31152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:12.438825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36197 (local address 127.15.232.130:54599)
0808 20:57:12.439019 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:12.439071 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:12.439612 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:12.439880 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:12.439883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:12.439891 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:12.440925 (+ 1034us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:57:12.440926 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:12.440955 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:57:12.440984 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:57:12.441248 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:57:12.441255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:57:12.441258 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:57:12.441341 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 20:57:12.441421 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260808 20:57:12.442221 31150 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36197
I20260808 20:57:12.442377 31150 heartbeater.cc:464] Registering TS with master...
I20260808 20:57:12.442512 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36197
--builtin_ntp_servers=127.15.232.148:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 20:57:12.442584 31150 heartbeater.cc:511] Master 127.15.232.190:36197 requested a full tablet report, sending...
I20260808 20:57:12.443061 30697 ts_manager.cc:194] Registered new tserver with Master: 63387e83b8cc42f180cac1c08ee15fcc (127.15.232.130:42493)
W20260808 20:57:12.529742 31153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:12.529953 31153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:12.529976 31153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:12.531152 31153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:12.531190 31153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:12.531215 31153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:57:12.532519 31153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:12.532708 31153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:12.532938 31153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:12.538182 31165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:12.538272 31160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:12.538347 31162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:12.538363 31153 server_base.cc:1034] running on GCE node
I20260808 20:57:12.538664 31153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:12.538920 31153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:12.540092 31153 hybrid_clock.cc:648] HybridClock initialized: now 1786222632540086 us; error 36 us; skew 500 ppm
I20260808 20:57:12.541381 31153 webserver.cc:460] Webserver started at http://127.15.232.131:34391/ using document root <none> and password file <none>
I20260808 20:57:12.541638 31153 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:12.541708 31153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:12.541863 31153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:12.543093 31153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "635e00fb556c4fec977e3df63954c802"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "9843977efb0ea1dedfebb2dbc1722271"
server_key_iv: "33fd6b6d1b69e419216b4a1aa2ffb0b0"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.543594 31153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "635e00fb556c4fec977e3df63954c802"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "9843977efb0ea1dedfebb2dbc1722271"
server_key_iv: "33fd6b6d1b69e419216b4a1aa2ffb0b0"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.545023 31153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:57:12.545869 31174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.546056 31153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:57:12.546164 31153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "635e00fb556c4fec977e3df63954c802"
format_stamp: "Formatted at 2026-08-08 20:57:12 on dist-test-slave-c12x"
server_key: "9843977efb0ea1dedfebb2dbc1722271"
server_key_iv: "33fd6b6d1b69e419216b4a1aa2ffb0b0"
server_key_version: "encryptionkey@0"
I20260808 20:57:12.546250 31153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:12.549746 31153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:12.550014 31153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:12.550158 31153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:12.550375 31153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:57:12.550695 31153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:57:12.550750 31153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.550791 31153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:57:12.550818 31153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:12.556790 31153 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:37807
I20260808 20:57:12.556833 31339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:37807 every 8 connection(s)
I20260808 20:57:12.557214 31153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:57:12.558749 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 31153
I20260808 20:57:12.558846 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:57:12.559087 16290 external_mini_cluster.cc:1428] Setting key b269bd54d1248bf4f5c198f1eb58085b
I20260808 20:57:12.560648 30748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:12.557967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:56765 (local address 127.15.232.190:36197)
0808 20:57:12.558029 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:12.558031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:12.558336 (+ 305us) server_negotiation.cc:407] Connection header received
0808 20:57:12.558992 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:12.558993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:12.558999 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:12.559079 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:12.560442 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:12.560443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:12.560444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 20:57:12.560447 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:12.560462 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 20:57:12.560466 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:12.560509 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:12.560586 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 20:57:12.560597 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 20:57:12.560846 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:12.557983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36197 (local address 127.15.232.131:56765)
0808 20:57:12.558205 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:12.558247 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:12.558883 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:12.559180 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:12.559184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:12.559196 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:12.560308 (+ 1112us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 20:57:12.560309 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:12.560339 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 20:57:12.560369 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:57:12.560525 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:57:12.560529 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:57:12.560531 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 20:57:12.560592 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 20:57:12.560643 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":124,"threads_started":1}
I20260808 20:57:12.561264 31340 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36197
I20260808 20:57:12.561347 31340 heartbeater.cc:464] Registering TS with master...
I20260808 20:57:12.561525 31340 heartbeater.cc:511] Master 127.15.232.190:36197 requested a full tablet report, sending...
I20260808 20:57:12.561869 30697 ts_manager.cc:194] Registered new tserver with Master: 635e00fb556c4fec977e3df63954c802 (127.15.232.131:37807)
I20260808 20:57:12.571259 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 20:57:12.573333 30748 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:57:12.572879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43806 (local address 127.15.232.190:36197)
0808 20:57:12.572926 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:12.572929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:12.572976 (+ 47us) server_negotiation.cc:407] Connection header received
0808 20:57:12.573036 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:12.573036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:12.573041 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:12.573096 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:12.573225 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43806: BlockingRecv error: recv got EOF from 127.0.0.1:43806 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 20:57:12.573431 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 30781
I20260808 20:57:12.587503 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 30959
I20260808 20:57:12.601529 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 31153
I20260808 20:57:12.621043 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 30656
2026-08-08T20:57:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (604 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:57:12 dist-test-slave-c12x krb5kdc[31353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:57:12 dist-test-slave-c12x krb5kdc[31353](info): set up 2 sockets
Aug 08 20:57:12 dist-test-slave-c12x krb5kdc[31353](info): commencing operation
krb5kdc: starting...
W20260808 20:57:13.857061 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:57:13 dist-test-slave-c12x krb5kdc[31353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:57:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:57:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:57:13.990994 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:32873
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:34061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:32873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:57:14.079967 31368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:14.080193 31368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:14.080216 31368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:14.081353 31368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:57:14.081389 31368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:14.081403 31368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:14.081415 31368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:57:14.081427 31368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:57:14.082890 31368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:34061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:32873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:32873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:14.083093 31368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:14.083269 31368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:14.088780 31377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:14.088780 31376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:14.088828 31380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:14.089205 31368 server_base.cc:1034] running on GCE node
I20260808 20:57:14.089445 31368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:14.089807 31368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:14.090971 31368 hybrid_clock.cc:648] HybridClock initialized: now 1786222634090964 us; error 47 us; skew 500 ppm
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[31353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:57:14.098155 31368 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:57:14.098598 31368 webserver.cc:460] Webserver started at http://127.15.232.190:39253/ using document root <none> and password file <none>
I20260808 20:57:14.098789 31368 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:14.098886 31368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:14.099015 31368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:14.100155 31368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "41d830673c114a24acb5d582c85853fb"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "652dc73297acfc16b27b1f07b3e18ab5"
server_key_iv: "fb3eaef5e8a9a7ef0430e2b8366e3f9b"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.100477 31368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "41d830673c114a24acb5d582c85853fb"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "652dc73297acfc16b27b1f07b3e18ab5"
server_key_iv: "fb3eaef5e8a9a7ef0430e2b8366e3f9b"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.101800 31368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:57:14.102523 31394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.102728 31368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:14.102823 31368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "41d830673c114a24acb5d582c85853fb"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "652dc73297acfc16b27b1f07b3e18ab5"
server_key_iv: "fb3eaef5e8a9a7ef0430e2b8366e3f9b"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.102898 31368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:14.112049 31368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:14.114384 31368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:14.114503 31368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:14.119006 31368 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:32873
I20260808 20:57:14.119190 31460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:32873 every 8 connection(s)
I20260808 20:57:14.119547 31368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:57:14.120700 31462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:57:14.123208 31462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb: Bootstrap starting.
I20260808 20:57:14.123869 31462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb: Neither blocks nor log segments found. Creating new log.
I20260808 20:57:14.124119 31462 log.cc:824] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb: Log is configured to *not* fsync() on all Append() calls
I20260808 20:57:14.124891 31462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb: No bootstrap required, opened a new log
I20260808 20:57:14.126483 31462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41d830673c114a24acb5d582c85853fb" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32873 } }
I20260808 20:57:14.126662 31462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:57:14.126688 31462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41d830673c114a24acb5d582c85853fb, State: Initialized, Role: FOLLOWER
I20260808 20:57:14.126799 31462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41d830673c114a24acb5d582c85853fb" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32873 } }
I20260808 20:57:14.126890 31462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:57:14.126928 31462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:57:14.127002 31462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:57:14.127272 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 31368
I20260808 20:57:14.127360 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:57:14.127595 16290 external_mini_cluster.cc:1428] Setting key 4f07ed18bd86d63c9851352d99cba09f
I20260808 20:57:14.128141 31462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41d830673c114a24acb5d582c85853fb" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32873 } }
I20260808 20:57:14.128270 31462 leader_election.cc:302] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41d830673c114a24acb5d582c85853fb; no voters:
I20260808 20:57:14.128469 31462 leader_election.cc:288] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:57:14.128492 31468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:57:14.128654 31468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [term 1 LEADER]: Becoming Leader. State: Replica: 41d830673c114a24acb5d582c85853fb, State: Running, Role: LEADER
I20260808 20:57:14.128741 31462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:57:14.128751 31468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41d830673c114a24acb5d582c85853fb" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32873 } }
I20260808 20:57:14.129161 31468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41d830673c114a24acb5d582c85853fb. Latest consensus state: current_term: 1 leader_uuid: "41d830673c114a24acb5d582c85853fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41d830673c114a24acb5d582c85853fb" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32873 } } }
I20260808 20:57:14.129262 31468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [sys.catalog]: This master's current role is: LEADER
I20260808 20:57:14.129172 31470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41d830673c114a24acb5d582c85853fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41d830673c114a24acb5d582c85853fb" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32873 } } }
I20260808 20:57:14.129418 31470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb [sys.catalog]: This master's current role is: LEADER
I20260808 20:57:14.129526 31476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:57:14.130015 31476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:57:14.132715 31476 catalog_manager.cc:1347] Generated new cluster ID: 145f74fdd89d405190a5eb68e7e6b798
I20260808 20:57:14.132776 31476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[31353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:57:14.139714 31476 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:57:14.140432 31476 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:57:14.140843 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.128139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.15.232.190:32873)
0808 20:57:14.128411 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.128423 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.128457 (+ 34us) server_negotiation.cc:407] Connection header received
0808 20:57:14.128588 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.128597 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.128656 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.129730 (+ 1074us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:14.129839 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.131492 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.132568 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.133190 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.133306 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.136230 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:14.136235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:14.136235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:57:14.136247 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:14.139059 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.139528 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.139531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.139532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.139656 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.139883 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.139884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.139885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.140128 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:14.140333 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.140401 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.140649 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":125,"threads_started":1}
I20260808 20:57:14.144409 31476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb: Generated new TSK 0
I20260808 20:57:14.144618 31476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:57:14.179139 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32873
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:34061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:57:14.265765 31499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:14.266037 31499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:14.266083 31499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:14.267308 31499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:14.267385 31499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:14.267433 31499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:57:14.269076 31499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:34061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:14.269395 31499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:14.269649 31499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:14.275178 31511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:14.275183 31506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:14.275220 31507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:14.275521 31499 server_base.cc:1034] running on GCE node
I20260808 20:57:14.275708 31499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:14.275955 31499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:14.277132 31499 hybrid_clock.cc:648] HybridClock initialized: now 1786222634277123 us; error 42 us; skew 500 ppm
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[31353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:57:14.279795 31499 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:57:14.280241 31499 webserver.cc:460] Webserver started at http://127.15.232.129:36813/ using document root <none> and password file <none>
I20260808 20:57:14.280491 31499 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:14.280568 31499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:14.280721 31499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:14.281798 31499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "e94d4bcca03346579e4c8c5ffb0671c1"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "f41c0f7c67b53cf31401b172f4094566"
server_key_iv: "d14d7611331ebb00f01ce10d2dea909d"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.282177 31499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "e94d4bcca03346579e4c8c5ffb0671c1"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "f41c0f7c67b53cf31401b172f4094566"
server_key_iv: "d14d7611331ebb00f01ce10d2dea909d"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.283756 31499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 20:57:14.284538 31520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.284768 31499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:14.284886 31499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "e94d4bcca03346579e4c8c5ffb0671c1"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "f41c0f7c67b53cf31401b172f4094566"
server_key_iv: "d14d7611331ebb00f01ce10d2dea909d"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.284976 31499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:14.296578 31499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:14.298580 31499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:14.298753 31499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:14.299001 31499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:57:14.299324 31499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:57:14.299379 31499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.299424 31499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:57:14.299543 31499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.305668 31499 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:36579
I20260808 20:57:14.305749 31681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:36579 every 8 connection(s)
I20260808 20:57:14.306087 31499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[31353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:57:14.315682 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 31499
I20260808 20:57:14.315802 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:57:14.315932 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.306815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:51873 (local address 127.15.232.190:32873)
0808 20:57:14.306880 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.306883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.307164 (+ 281us) server_negotiation.cc:407] Connection header received
0808 20:57:14.307900 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.307901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.307915 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.307980 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:14.308971 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.309504 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.310496 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.310716 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.310808 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.313065 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:14.313066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:14.313066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:57:14.313073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:14.314372 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.314872 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.314873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.314873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.314930 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.315170 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.315170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.315170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.315279 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:14.315331 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.315727 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.315849 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 20:57:14.316039 16290 external_mini_cluster.cc:1428] Setting key de3625564d9f16d93e2b9b58de236f4c
I20260808 20:57:14.316140 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.306821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32873 (local address 127.15.232.129:51873)
0808 20:57:14.307038 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:14.307076 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:14.307794 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:14.308055 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:14.308060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:14.308671 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:14.308867 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.308871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.309612 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.309613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.310405 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.310407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.310822 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.310823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.310948 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.311501 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:14.311538 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:14.312964 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:57:14.314524 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:57:14.314532 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:57:14.314534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:57:14.314790 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:57:14.315000 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:57:14.315001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:57:14.315001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:57:14.315110 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:57:14.315336 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:57:14.315341 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:57:14.315581 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 20:57:14.315710 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 20:57:14.315899 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":123,"threads_started":1}
I20260808 20:57:14.316666 31683 heartbeater.cc:347] Connected to a master server at 127.15.232.190:32873
I20260808 20:57:14.316792 31683 heartbeater.cc:464] Registering TS with master...
I20260808 20:57:14.316977 31683 heartbeater.cc:511] Master 127.15.232.190:32873 requested a full tablet report, sending...
I20260808 20:57:14.317533 31415 ts_manager.cc:194] Registered new tserver with Master: e94d4bcca03346579e4c8c5ffb0671c1 (127.15.232.129:36579)
I20260808 20:57:14.318149 31415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:51873
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:57:14.342849 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32873
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:34061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:57:14.429764 31690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:14.429980 31690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:14.430006 31690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:14.431262 31690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:14.431313 31690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:14.431348 31690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:57:14.432894 31690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:34061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:14.433162 31690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:14.433404 31690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:14.439154 31699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:14.439208 31697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:14.439185 31701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:14.439272 31690 server_base.cc:1034] running on GCE node
I20260808 20:57:14.439631 31690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:14.440044 31690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:14.441244 31690 hybrid_clock.cc:648] HybridClock initialized: now 1786222634441226 us; error 47 us; skew 500 ppm
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[31353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:57:14.444067 31690 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:57:14.444582 31690 webserver.cc:460] Webserver started at http://127.15.232.130:34979/ using document root <none> and password file <none>
I20260808 20:57:14.444844 31690 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:14.444934 31690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:14.445097 31690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:14.446239 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "fc6ee52e5e6446dab555c8f0afb47f8a"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "fea4b4212906c31b58b831a4ee1e4a82"
server_key_iv: "6c72bf0d522dbc68d3f29cf181ffb858"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.446646 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "fc6ee52e5e6446dab555c8f0afb47f8a"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "fea4b4212906c31b58b831a4ee1e4a82"
server_key_iv: "6c72bf0d522dbc68d3f29cf181ffb858"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.448192 31690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:57:14.449120 31711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.449312 31690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:57:14.449427 31690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "fc6ee52e5e6446dab555c8f0afb47f8a"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "fea4b4212906c31b58b831a4ee1e4a82"
server_key_iv: "6c72bf0d522dbc68d3f29cf181ffb858"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.449508 31690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:14.467679 31690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:14.469961 31690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:14.470139 31690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:14.470395 31690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:57:14.470736 31690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:57:14.470793 31690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.470834 31690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:57:14.470880 31690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.477337 31690 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:44425
I20260808 20:57:14.477380 31871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:44425 every 8 connection(s)
I20260808 20:57:14.477739 31690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:57:14.479233 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 31690
I20260808 20:57:14.479354 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:57:14.479573 16290 external_mini_cluster.cc:1428] Setting key d48e9e0b032ce93172921b8ec43460a8
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[31353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:57:14.487665 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.478533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:47407 (local address 127.15.232.190:32873)
0808 20:57:14.478611 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.478614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.478829 (+ 215us) server_negotiation.cc:407] Connection header received
0808 20:57:14.479360 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.479361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.479374 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.479475 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:14.480542 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.481035 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.482071 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.482265 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.482346 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.484638 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:14.484641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:14.484641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:57:14.484649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:14.486110 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.486589 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.486591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.486591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.486655 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.486942 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.486943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.486943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.487067 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:14.487151 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.487479 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.487573 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 20:57:14.487970 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.478510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32873 (local address 127.15.232.130:47407)
0808 20:57:14.478702 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:14.478742 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:14.479242 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:14.479534 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:14.479539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:14.480218 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:14.480414 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.480419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.481133 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.481134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.481973 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.481975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.482353 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.482355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.482499 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.483071 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:14.483079 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:14.484530 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:57:14.486253 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:57:14.486261 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:57:14.486263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:57:14.486507 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:57:14.486735 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:57:14.486736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:57:14.486736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:57:14.486862 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:57:14.487141 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:57:14.487147 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:57:14.487299 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 20:57:14.487405 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 20:57:14.487597 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260808 20:57:14.488507 31872 heartbeater.cc:347] Connected to a master server at 127.15.232.190:32873
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
I20260808 20:57:14.488626 31872 heartbeater.cc:464] Registering TS with master...
I20260808 20:57:14.488816 31872 heartbeater.cc:511] Master 127.15.232.190:32873 requested a full tablet report, sending...
I20260808 20:57:14.489239 31415 ts_manager.cc:194] Registered new tserver with Master: fc6ee52e5e6446dab555c8f0afb47f8a (127.15.232.130:44425)
I20260808 20:57:14.489737 31415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:47407
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:57:14.510757 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32873
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:34061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:57:14.601413 31879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:57:14.601651 31879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:57:14.601678 31879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:57:14.602849 31879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:57:14.602893 31879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:57:14.602921 31879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:57:14.604437 31879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:34061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:57:14.604665 31879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:57:14.604859 31879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:57:14.610357 31892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:14.610348 31890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:57:14.610348 31888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:57:14.610831 31879 server_base.cc:1034] running on GCE node
I20260808 20:57:14.611052 31879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:57:14.611267 31879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:57:14.612449 31879 hybrid_clock.cc:648] HybridClock initialized: now 1786222634612436 us; error 44 us; skew 500 ppm
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[31353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:57:14.615356 31879 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:57:14.615892 31879 webserver.cc:460] Webserver started at http://127.15.232.131:33663/ using document root <none> and password file <none>
I20260808 20:57:14.616154 31879 fs_manager.cc:359] Metadata directory not provided
I20260808 20:57:14.616233 31879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:57:14.616389 31879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:57:14.617503 31879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "1501d047f38644c2a6de447c7319e39b"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "566b10d6924a72784816b5adf9483961"
server_key_iv: "f279403f8fa0e3c511aa5827714fdbcd"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.617897 31879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "1501d047f38644c2a6de447c7319e39b"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "566b10d6924a72784816b5adf9483961"
server_key_iv: "f279403f8fa0e3c511aa5827714fdbcd"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.619385 31879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260808 20:57:14.620524 31901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.620879 31879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:57:14.621129 31879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "1501d047f38644c2a6de447c7319e39b"
format_stamp: "Formatted at 2026-08-08 20:57:14 on dist-test-slave-c12x"
server_key: "566b10d6924a72784816b5adf9483961"
server_key_iv: "f279403f8fa0e3c511aa5827714fdbcd"
server_key_version: "encryptionkey@0"
I20260808 20:57:14.621263 31879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:57:14.632874 31879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:57:14.635293 31879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:57:14.635526 31879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:57:14.635780 31879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:57:14.636125 31879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:57:14.636181 31879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.636225 31879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:57:14.636270 31879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:57:14.642469 31879 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:37337
I20260808 20:57:14.642526 32061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:37337 every 8 connection(s)
I20260808 20:57:14.642859 31879 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:57:14.647681 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 31879
I20260808 20:57:14.647801 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:57:14.647993 16290 external_mini_cluster.cc:1428] Setting key 7c413afcb8605852623c9f87d362134b
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[31353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:57:14.653142 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.643719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:59929 (local address 127.15.232.190:32873)
0808 20:57:14.643767 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.643769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.644009 (+ 240us) server_negotiation.cc:407] Connection header received
0808 20:57:14.644563 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.644565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.644577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.644646 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:14.645691 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.646209 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.647266 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.647587 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.647677 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.650195 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:14.650196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:14.650197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:57:14.650213 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:14.651602 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.652034 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.652035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.652036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.652108 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.652400 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.652402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.652402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.652556 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:14.652645 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.652938 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.653028 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 20:57:14.653357 32065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.643722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32873 (local address 127.15.232.131:59929)
0808 20:57:14.643879 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:14.643915 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:14.644444 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:14.644727 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:14.644730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:14.645382 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:57:14.645583 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.645588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.646330 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.646331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.647181 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.647183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.647679 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.647680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.647828 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.648482 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:57:14.648492 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:57:14.650066 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:57:14.651689 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:57:14.651697 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:57:14.651700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:57:14.651946 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:57:14.652188 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:57:14.652189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:57:14.652189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:57:14.652321 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:57:14.652634 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:57:14.652642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:57:14.652798 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 20:57:14.652916 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 20:57:14.653090 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260808 20:57:14.653932 32062 heartbeater.cc:347] Connected to a master server at 127.15.232.190:32873
I20260808 20:57:14.654032 32062 heartbeater.cc:464] Registering TS with master...
I20260808 20:57:14.654229 32062 heartbeater.cc:511] Master 127.15.232.190:32873 requested a full tablet report, sending...
I20260808 20:57:14.654700 31415 ts_manager.cc:194] Registered new tserver with Master: 1501d047f38644c2a6de447c7319e39b (127.15.232.131:37337)
I20260808 20:57:14.655061 31415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:59929
I20260808 20:57:14.660218 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:57:14.668277 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.661644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55196 (local address 127.15.232.190:32873)
0808 20:57:14.661728 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.661731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.661846 (+ 115us) server_negotiation.cc:407] Connection header received
0808 20:57:14.661953 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.661954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.661971 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.662033 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:57:14.662627 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.663130 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.663922 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.664122 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.664211 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.665607 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:57:14.665608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:57:14.665609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:57:14.665616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:57:14.666955 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.667359 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.667360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.667360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.667426 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:57:14.667666 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:57:14.667667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:57:14.667667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:57:14.667836 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:57:14.667959 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.668078 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.668162 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 20:57:14.670307 31415 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:57:14.672071 31415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:57:14.679557 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.675063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47874 (local address 127.15.232.129:36579)
0808 20:57:14.675212 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.675217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.675293 (+ 76us) server_negotiation.cc:407] Connection header received
0808 20:57:14.675377 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.675380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.675397 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.675525 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:57:14.675736 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.677080 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.678416 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.679097 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.679165 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.679223 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.679407 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.679472 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260808 20:57:14.679567 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.674752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:44425 (local address 127.0.0.1:38106)
0808 20:57:14.674922 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:14.675073 (+ 151us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:14.675154 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:14.675858 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:14.675859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:14.675861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:57:14.675909 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.675911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.677313 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.677314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.678222 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.678225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.679184 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.679185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.679278 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.679294 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 20:57:14.679376 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 20:57:14.679469 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":30,"thread_start_us":65,"threads_started":1}
I20260808 20:57:14.679963 32083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.674955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.15.232.130:44425)
0808 20:57:14.675629 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.675633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.675643 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:57:14.675682 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.675684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.675700 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.675775 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:57:14.676004 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.677213 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.678315 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.679068 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.679158 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.679223 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.679831 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.679880 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":105,"threads_started":1}
I20260808 20:57:14.680013 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.674752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:36579 (local address 127.0.0.1:47874)
0808 20:57:14.675234 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:14.675244 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:14.675307 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:14.675541 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:14.675544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:14.675547 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:57:14.675632 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.675635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.677194 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.677195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.678336 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.678339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.679187 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.679187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.679281 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.679294 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 20:57:14.679376 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 20:57:14.679469 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":41,"thread_start_us":95,"threads_started":1}
I20260808 20:57:14.680759 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.674774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:37337 (local address 127.0.0.1:51360)
0808 20:57:14.675394 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:14.675403 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:14.675497 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:14.675893 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:14.675894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:14.675896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:57:14.675939 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.675941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.676970 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.676971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.678338 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.678339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.680464 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.680465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.680526 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.680530 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:57:14.680592 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 20:57:14.680634 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260808 20:57:14.680728 31589 tablet_service.cc:1467] Processing CreateTablet for tablet f49e85d68d004dbfa82fadc7f5bef53a (DEFAULT_TABLE table=test-table [id=2e602a7e270446838d605f418115632e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:57:14.680733 31780 tablet_service.cc:1467] Processing CreateTablet for tablet f49e85d68d004dbfa82fadc7f5bef53a (DEFAULT_TABLE table=test-table [id=2e602a7e270446838d605f418115632e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:57:14.680980 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.675086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.15.232.131:37337)
0808 20:57:14.675303 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.675312 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.675463 (+ 151us) server_negotiation.cc:407] Connection header received
0808 20:57:14.675763 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.675765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.675782 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.675893 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:57:14.676000 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.676668 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.679272 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.680307 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.680375 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.680429 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.680855 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.680905 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":100,"threads_started":1}
I20260808 20:57:14.681090 31589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f49e85d68d004dbfa82fadc7f5bef53a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:57:14.681113 31780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f49e85d68d004dbfa82fadc7f5bef53a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:57:14.684105 32087 tablet_bootstrap.cc:492] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1: Bootstrap starting.
I20260808 20:57:14.684840 32087 tablet_bootstrap.cc:654] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1: Neither blocks nor log segments found. Creating new log.
I20260808 20:57:14.684803 31973 tablet_service.cc:1467] Processing CreateTablet for tablet f49e85d68d004dbfa82fadc7f5bef53a (DEFAULT_TABLE table=test-table [id=2e602a7e270446838d605f418115632e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:57:14.685149 31973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f49e85d68d004dbfa82fadc7f5bef53a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:57:14.685472 32087 log.cc:824] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1: Log is configured to *not* fsync() on all Append() calls
I20260808 20:57:14.685809 32088 tablet_bootstrap.cc:492] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: Bootstrap starting.
I20260808 20:57:14.686326 32087 tablet_bootstrap.cc:492] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1: No bootstrap required, opened a new log
I20260808 20:57:14.686480 32087 ts_tablet_manager.cc:1397] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:57:14.686527 32088 tablet_bootstrap.cc:654] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: Neither blocks nor log segments found. Creating new log.
I20260808 20:57:14.687120 32088 log.cc:824] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: Log is configured to *not* fsync() on all Append() calls
I20260808 20:57:14.687956 32088 tablet_bootstrap.cc:492] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: No bootstrap required, opened a new log
I20260808 20:57:14.688042 32088 ts_tablet_manager.cc:1397] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:57:14.688423 32095 tablet_bootstrap.cc:492] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: Bootstrap starting.
I20260808 20:57:14.688892 32087 raft_consensus.cc:348] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } }
I20260808 20:57:14.689035 32095 tablet_bootstrap.cc:654] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: Neither blocks nor log segments found. Creating new log.
I20260808 20:57:14.689074 32087 raft_consensus.cc:374] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:57:14.689136 32087 raft_consensus.cc:729] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e94d4bcca03346579e4c8c5ffb0671c1, State: Initialized, Role: FOLLOWER
I20260808 20:57:14.689260 32087 consensus_queue.cc:260] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } }
I20260808 20:57:14.689574 32087 ts_tablet_manager.cc:1428] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 20:57:14.689675 32095 log.cc:824] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: Log is configured to *not* fsync() on all Append() calls
I20260808 20:57:14.689716 31683 heartbeater.cc:503] Master 127.15.232.190:32873 was elected leader, sending a full tablet report...
I20260808 20:57:14.690480 32095 tablet_bootstrap.cc:492] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: No bootstrap required, opened a new log
I20260808 20:57:14.690564 32095 ts_tablet_manager.cc:1397] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:57:14.690675 32088 raft_consensus.cc:348] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } }
I20260808 20:57:14.690833 32088 raft_consensus.cc:374] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:57:14.690869 32088 raft_consensus.cc:729] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6ee52e5e6446dab555c8f0afb47f8a, State: Initialized, Role: FOLLOWER
I20260808 20:57:14.690990 32088 consensus_queue.cc:260] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } }
I20260808 20:57:14.691277 32088 ts_tablet_manager.cc:1428] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 20:57:14.691306 31872 heartbeater.cc:503] Master 127.15.232.190:32873 was elected leader, sending a full tablet report...
I20260808 20:57:14.692418 32095 raft_consensus.cc:348] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } }
I20260808 20:57:14.692552 32095 raft_consensus.cc:374] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:57:14.692574 32095 raft_consensus.cc:729] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1501d047f38644c2a6de447c7319e39b, State: Initialized, Role: FOLLOWER
I20260808 20:57:14.692684 32095 consensus_queue.cc:260] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } }
I20260808 20:57:14.692970 32095 ts_tablet_manager.cc:1428] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 20:57:14.693007 32062 heartbeater.cc:503] Master 127.15.232.190:32873 was elected leader, sending a full tablet report...
I20260808 20:57:14.720489 32102 raft_consensus.cc:482] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:57:14.720690 32102 raft_consensus.cc:504] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } }
I20260808 20:57:14.721172 32102 leader_election.cc:288] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc6ee52e5e6446dab555c8f0afb47f8a (127.15.232.130:44425), e94d4bcca03346579e4c8c5ffb0671c1 (127.15.232.129:36579)
I20260808 20:57:14.723991 32083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.721404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:42965 (local address 127.15.232.130:44425)
0808 20:57:14.721462 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.721464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.721474 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:57:14.721501 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.721502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.721511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.721566 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:57:14.721764 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.722247 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.723189 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.723631 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.723713 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.723723 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.723884 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.723918 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 20:57:14.723991 32065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.721336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:44425 (local address 127.15.232.131:42965)
0808 20:57:14.721383 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:14.721395 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:14.721437 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:14.721626 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:14.721627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:14.721629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:57:14.721702 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.721704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.722340 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.722341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.723086 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.723088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.723721 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.723722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.723809 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.723812 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:57:14.723879 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 20:57:14.723919 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 20:57:14.724282 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.721337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:36579 (local address 127.15.232.131:33733)
0808 20:57:14.721587 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:14.721597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:57:14.721648 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:57:14.721798 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:57:14.721800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:57:14.721802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:57:14.721875 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.721877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.722563 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.722564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.723384 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:57:14.723387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.724013 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.724014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:57:14.724094 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.724099 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:57:14.724157 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 20:57:14.724201 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":42,"thread_start_us":118,"threads_started":1}
I20260808 20:57:14.724323 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.721516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:33733 (local address 127.15.232.129:36579)
0808 20:57:14.721570 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.721572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.721659 (+ 87us) server_negotiation.cc:407] Connection header received
0808 20:57:14.721706 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.721707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.721714 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.721792 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:57:14.721983 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.722464 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.723549 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.723923 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.724015 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.724025 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.724179 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.724251 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:57:14.724375 31811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f49e85d68d004dbfa82fadc7f5bef53a" candidate_uuid: "1501d047f38644c2a6de447c7319e39b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" is_pre_election: true
I20260808 20:57:14.724521 31811 raft_consensus.cc:2454] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1501d047f38644c2a6de447c7319e39b in term 0.
I20260808 20:57:14.724709 31618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f49e85d68d004dbfa82fadc7f5bef53a" candidate_uuid: "1501d047f38644c2a6de447c7319e39b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" is_pre_election: true
I20260808 20:57:14.724820 31905 leader_election.cc:302] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1501d047f38644c2a6de447c7319e39b, fc6ee52e5e6446dab555c8f0afb47f8a; no voters:
I20260808 20:57:14.724864 31618 raft_consensus.cc:2454] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1501d047f38644c2a6de447c7319e39b in term 0.
I20260808 20:57:14.724948 32102 raft_consensus.cc:2781] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:57:14.725039 32102 raft_consensus.cc:482] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:57:14.725075 32102 raft_consensus.cc:3037] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:57:14.725922 32102 raft_consensus.cc:504] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } }
I20260808 20:57:14.726079 32102 leader_election.cc:288] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [CANDIDATE]: Term 1 election: Requested vote from peers fc6ee52e5e6446dab555c8f0afb47f8a (127.15.232.130:44425), e94d4bcca03346579e4c8c5ffb0671c1 (127.15.232.129:36579)
I20260808 20:57:14.726239 31811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f49e85d68d004dbfa82fadc7f5bef53a" candidate_uuid: "1501d047f38644c2a6de447c7319e39b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a"
I20260808 20:57:14.726321 31811 raft_consensus.cc:3037] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:57:14.727108 31811 raft_consensus.cc:2454] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1501d047f38644c2a6de447c7319e39b in term 1.
I20260808 20:57:14.727229 31618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f49e85d68d004dbfa82fadc7f5bef53a" candidate_uuid: "1501d047f38644c2a6de447c7319e39b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e94d4bcca03346579e4c8c5ffb0671c1"
I20260808 20:57:14.727346 31905 leader_election.cc:302] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1501d047f38644c2a6de447c7319e39b, fc6ee52e5e6446dab555c8f0afb47f8a; no voters:
I20260808 20:57:14.727337 31618 raft_consensus.cc:3037] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:57:14.728190 31618 raft_consensus.cc:2454] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1501d047f38644c2a6de447c7319e39b in term 1.
W20260808 20:57:14.728232 31874 tablet.cc:2369] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:57:14.728299 32102 raft_consensus.cc:2781] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:57:14.728466 32102 raft_consensus.cc:686] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 1 LEADER]: Becoming Leader. State: Replica: 1501d047f38644c2a6de447c7319e39b, State: Running, Role: LEADER
I20260808 20:57:14.728578 32102 consensus_queue.cc:237] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } }
I20260808 20:57:14.729506 31415 catalog_manager.cc:5495] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1501d047f38644c2a6de447c7319e39b (127.15.232.131). New cstate: current_term: 1 leader_uuid: "1501d047f38644c2a6de447c7319e39b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1501d047f38644c2a6de447c7319e39b" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 37337 } health_report { overall_health: HEALTHY } } }
I20260808 20:57:14.737593 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.734770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51372 (local address 127.15.232.131:37337)
0808 20:57:14.734829 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 20:57:14.734831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:57:14.734846 (+ 15us) server_negotiation.cc:407] Connection header received
0808 20:57:14.734951 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:57:14.734952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:57:14.734982 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:57:14.735066 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:57:14.735236 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.735852 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.736702 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:57:14.736908 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:57:14.736993 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:57:14.737179 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:57:14.737295 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:57:14.737413 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 20:57:14.737436 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 20:57:14.737461 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 20:57:14.739835 31811 raft_consensus.cc:1261] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [term 1 FOLLOWER]: Refusing update from remote peer 1501d047f38644c2a6de447c7319e39b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:57:14.739841 31618 raft_consensus.cc:1261] T f49e85d68d004dbfa82fadc7f5bef53a P e94d4bcca03346579e4c8c5ffb0671c1 [term 1 FOLLOWER]: Refusing update from remote peer 1501d047f38644c2a6de447c7319e39b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:57:14.740252 32106 consensus_queue.cc:1035] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6ee52e5e6446dab555c8f0afb47f8a" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 44425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:57:14.740379 32102 consensus_queue.cc:1035] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e94d4bcca03346579e4c8c5ffb0671c1" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:57:14.744058 32110 mvcc.cc:205] Tried to move back new op lower bound from 7316367911890481152 to 7316367911848996864. Current Snapshot: MvccSnapshot[applied={T|T < 7316367911890481152 or (T in {7316367911890481152})}]
I20260808 20:57:14.745249 31415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:57:14.745363 31415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:57:14.746182 31415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41d830673c114a24acb5d582c85853fb: Sending DeleteTablet for 3 replicas of tablet f49e85d68d004dbfa82fadc7f5bef53a
I20260808 20:57:14.746552 31973 tablet_service.cc:1514] Processing DeleteTablet for tablet f49e85d68d004dbfa82fadc7f5bef53a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:57:14 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:51360
I20260808 20:57:14.746630 31780 tablet_service.cc:1514] Processing DeleteTablet for tablet f49e85d68d004dbfa82fadc7f5bef53a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:57:14 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:38106
I20260808 20:57:14.746778 31589 tablet_service.cc:1514] Processing DeleteTablet for tablet f49e85d68d004dbfa82fadc7f5bef53a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:57:14 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:47874
I20260808 20:57:14.746935 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 31499
I20260808 20:57:14.747522 32122 tablet_replica.cc:330] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: stopping tablet replica
I20260808 20:57:14.747637 32122 raft_consensus.cc:2229] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:57:14.747789 32122 raft_consensus.cc:2258] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:57:14.749598 32122 ts_tablet_manager.cc:1905] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:57:14.751456 32124 tablet_replica.cc:330] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: stopping tablet replica
I20260808 20:57:14.751567 32124 raft_consensus.cc:2229] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:57:14.751669 32124 raft_consensus.cc:2258] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:57:14.753413 32122 ts_tablet_manager.cc:1918] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:57:14.753543 32122 log.cc:1196] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/f49e85d68d004dbfa82fadc7f5bef53a
I20260808 20:57:14.753858 32122 ts_tablet_manager.cc:1939] T f49e85d68d004dbfa82fadc7f5bef53a P 1501d047f38644c2a6de447c7319e39b: Deleting consensus metadata
I20260808 20:57:14.754384 31395 catalog_manager.cc:4841] TS 1501d047f38644c2a6de447c7319e39b (127.15.232.131:37337): tablet f49e85d68d004dbfa82fadc7f5bef53a (table test-table [id=2e602a7e270446838d605f418115632e]) successfully deleted
I20260808 20:57:14.756031 32124 ts_tablet_manager.cc:1905] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:57:14.757490 32124 ts_tablet_manager.cc:1918] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:57:14.757602 32124 log.cc:1196] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/f49e85d68d004dbfa82fadc7f5bef53a
I20260808 20:57:14.757959 32124 ts_tablet_manager.cc:1939] T f49e85d68d004dbfa82fadc7f5bef53a P fc6ee52e5e6446dab555c8f0afb47f8a: Deleting consensus metadata
W20260808 20:57:14.757982 31399 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:36579 (error 108)
I20260808 20:57:14.758484 31397 catalog_manager.cc:4841] TS fc6ee52e5e6446dab555c8f0afb47f8a (127.15.232.130:44425): tablet f49e85d68d004dbfa82fadc7f5bef53a (table test-table [id=2e602a7e270446838d605f418115632e]) successfully deleted
I20260808 20:57:14.758567 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:57:14.758425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:36579 (local address 127.0.0.1:47880)
0808 20:57:14.758492 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 20:57:14.758535 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:36579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 20:57:14.758633 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 31690
W20260808 20:57:14.758796 31399 catalog_manager.cc:4584] TS e94d4bcca03346579e4c8c5ffb0671c1 (127.15.232.129:36579): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f49e85d68d004dbfa82fadc7f5bef53a: Network error: Client connection negotiation failed: client connection to 127.15.232.129:36579: connect: Connection refused (error 111)
I20260808 20:57:14.770350 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 31879
I20260808 20:57:14.782915 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 31368
2026-08-08T20:57:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2168 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[32131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[32131](info): set up 2 sockets
Aug 08 20:57:14 dist-test-slave-c12x krb5kdc[32131](info): commencing operation
krb5kdc: starting...
W20260808 20:57:16.033030 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:57:16 dist-test-slave-c12x krb5kdc[32131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:57:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:57:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.232.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.232.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/rangeradmin_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/rangeradmin_127.15.232.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.232.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.232.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/rangerlookup_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/rangerlookup_127.15.232.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.232.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.232.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/HTTP_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/HTTP_127.15.232.148@KRBTEST.COM.keytab.
I20260808 20:57:16.222523 16290 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0tXHI2/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/postgres -l logfile start
2026-08-08 20:57:18.982 UTC [32171] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 20:57:18.982 UTC [32171] LOG: listening on IPv4 address "127.15.232.148", port 58397
2026-08-08 20:57:18.987 UTC [32171] LOG: listening on Unix socket "/tmp/.s.PGSQL.58397"
2026-08-08 20:57:19.020 UTC [32174] LOG: database system was shut down at 2026-08-08 20:57:17 UTC
2026-08-08 20:57:19.025 UTC [32171] LOG: database system is ready to accept connections
I20260808 20:57:20.162014 16290 mini_postgres.cc:96] Postgres bound to 58397
2026-08-08 20:57:20.171 UTC [32187] FATAL: database "slave" does not exist
127.15.232.148:58397 - accepting connections
I20260808 20:57:20.172103 16290 mini_ranger.cc:161] Starting Ranger...
I20260808 20:57:20.190591 16290 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 20:57:20.731503 16290 mini_ranger.cc:87] Created ranger Postgres database
I20260808 20:57:20.731606 16290 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-admin
2026-08-08 20:57:21,146 [I] DB FLAVOR :POSTGRES
2026-08-08 20:57:21,146 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 20:57:21,147 [I] Checking connection..
2026-08-08 20:57:21,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 20:57:21,422 [I] Checking connection passed.
2026-08-08 20:57:21,423 [I] --------- Verifying version history table ---------
2026-08-08 20:57:21,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 20:57:21,695 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 20:57:21,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 20:57:21,978 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 20:57:21,979 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 20:57:21,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 20:57:22.235 UTC [32278] WARNING: there is no transaction in progress
2026-08-08 20:57:22,245 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 20:57:22,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 20:57:22,502 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 20:57:22,502 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 20:57:22,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 20:57:22,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 20:57:23,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-c12x.c.gcp-upstream.internal','N') ;"
2026-08-08 20:57:23,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 20:57:23,534 [I] Table x_portal_user does not exist in database ranger
2026-08-08 20:57:23,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 20:57:23,806 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 20:57:23,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 20:57:24,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 20:57:24,348 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 20:57:24,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 20:57:25.412 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.430 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.448 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.465 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.482 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.499 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.556 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.564 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.580 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.591 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.602 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.617 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.629 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.636 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.643 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:25.650 UTC [32447] WARNING: there is no transaction in progress
2026-08-08 20:57:26,364 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 20:57:26,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 20:57:26,616 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 20:57:26,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 20:57:26,909 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 20:57:26,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 20:57:27,186 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 20:57:27,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 20:57:27,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 20:57:27,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-c12x.c.gcp-upstream.internal';"
2026-08-08 20:57:27,977 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 20:57:27,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 20:57:28,227 [I] DB_PATCHES have already been applied
I20260808 20:57:28.233140 16290 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-admin:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0tXHI2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 20:57:28.233215 16290 mini_ranger.cc:193] Using host: 127.15.232.148
I20260808 20:57:28.237272 16290 mini_ranger.cc:227] Ranger admin URL: http://127.15.232.148:45021
Aug 08, 2026 8:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 8:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 8:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 8:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 8:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 8:57:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 8:57:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 20:57:29.584717 16290 mini_ranger.cc:160] Time spent starting Ranger: real 9.413s user 0.001s sys 0.011s
Aug 08, 2026 8:57:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 8:57:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 8:57:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 8:57:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 8:57:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 20:57:53.038347 16290 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.232.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.232.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/rangerkms_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/rangerkms_127.15.232.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.232.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.232.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.232.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/HTTP_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.232.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/HTTP_127.15.232.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 20:57:53 dist-test-slave-c12x krb5kdc[32131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222673, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 20:57:53.133246 16290 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 20:57:53.147245 16290 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 20:57:54.122316 16290 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 20:57:54.122408 16290 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-kms
I20260808 20:57:54.122428 16290 mini_ranger_kms.cc:227] Using postgres at 127.15.232.148:58397
2026-08-08 20:57:54,696 [I] DB FLAVOR :POSTGRES
2026-08-08 20:57:54,697 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 20:57:54,697 [I] Checking connection
2026-08-08 20:57:54,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 20:57:54,978 [I] connection success
2026-08-08 20:57:54,978 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 20:57:54,978 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 20:57:54,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 20:57:55,294 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 20:57:55,294 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 20:57:55,295 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 20:57:55,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:58397/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 20:57:55,596 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 20:57:56.220718 16290 mini_ranger_kms.cc:310] Created kms service
I20260808 20:57:56.490660 16290 mini_ranger_kms.cc:326] Created kudu user
I20260808 20:57:56.543126 16290 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 20:57:56 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222673, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:57:57.010452 16290 mini_ranger_kms.cc:384] Added ranger policy
I20260808 20:57:57.010583 16290 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-kms:/tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 20:57:57.010604 16290 mini_ranger_kms.cc:240] Using host: 127.15.232.148
I20260808 20:57:57.015079 16290 mini_ranger_kms.cc:276] Ranger KMS PID: 32699
I20260808 20:57:57.015223 16290 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.232.148:37977
Aug 08, 2026 8:57:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 8:57:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 8:57:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 8:57:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 8:57:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 8:57:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 20:57:59 dist-test-slave-c12x krb5kdc[32131](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786222679, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:57:59.754529 16290 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.621s user 0.025s sys 0.316s
I20260808 20:57:59.754644 16290 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 20:57:59.754720 16290 mini_ranger_kms.cc:401] 127.15.232.148:37977/kms/v1/keys
Aug 08 20:57:59 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222679, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.148@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
Aug 08 20:58:00 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222679, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.148@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
Aug 08 20:58:02 dist-test-slave-c12x krb5kdc[32131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:58:02.885847 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37141
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:37141
--ranger_config_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:37977/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:58:02.981238 32763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:02.981534 32763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:02.981562 32763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:02.983040 32763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:58:02.983104 32763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:02.983120 32763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:02.983132 32763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:58:02.983146 32763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:58:02.985137 32763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.232.148:37977/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:37141
--ranger_config_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:02.985446 32763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:02.985735 32763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:02.991243 312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:02.991329 305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:02.991281 304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:02.992172 32763 server_base.cc:1034] running on GCE node
I20260808 20:58:02.992456 32763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:02.992913 32763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:02.994107 32763 hybrid_clock.cc:648] HybridClock initialized: now 1786222682994080 us; error 41 us; skew 500 ppm
Aug 08 20:58:02 dist-test-slave-c12x krb5kdc[32131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222682, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:03.002808 32763 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:58:03.003755 32763 webserver.cc:460] Webserver started at http://127.15.232.190:43317/ using document root <none> and password file <none>
I20260808 20:58:03.004082 32763 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:03.004150 32763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:03.004498 32763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 20:58:03 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222682, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:03.071624 32763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "c96db77d5feb4af19960b94a877229b5"
format_stamp: "Formatted at 2026-08-08 20:58:03 on dist-test-slave-c12x"
server_key: "7d657190533960be335eb3913d04f4cc"
server_key_iv: "c1e9ad17e1fde95f1b82fa71926b42ba"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:03.072208 32763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "c96db77d5feb4af19960b94a877229b5"
format_stamp: "Formatted at 2026-08-08 20:58:03 on dist-test-slave-c12x"
server_key: "7d657190533960be335eb3913d04f4cc"
server_key_iv: "c1e9ad17e1fde95f1b82fa71926b42ba"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:03.073936 32763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:58:03.089989 323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:03.090245 32763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:58:03.090353 32763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "c96db77d5feb4af19960b94a877229b5"
format_stamp: "Formatted at 2026-08-08 20:58:03 on dist-test-slave-c12x"
server_key: "7d657190533960be335eb3913d04f4cc"
server_key_iv: "c1e9ad17e1fde95f1b82fa71926b42ba"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:03.090466 32763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:03.102474 32763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:03.104477 32763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:03.104635 32763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:03.110584 32763 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:37141
I20260808 20:58:03.110606 392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:37141 every 8 connection(s)
I20260808 20:58:03.111047 32763 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:58:03.111940 393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:03.114408 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 32763
I20260808 20:58:03.114539 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:58:03.114763 16290 external_mini_cluster.cc:1428] Setting key 574f5bba79134a94197499bb172edee6
I20260808 20:58:03.115837 393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5: Bootstrap starting.
I20260808 20:58:03.116580 393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:03.117071 393 log.cc:824] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:03.118050 393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5: No bootstrap required, opened a new log
I20260808 20:58:03.120762 393 raft_consensus.cc:348] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96db77d5feb4af19960b94a877229b5" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37141 } }
I20260808 20:58:03.120954 393 raft_consensus.cc:374] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:03.120988 393 raft_consensus.cc:729] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96db77d5feb4af19960b94a877229b5, State: Initialized, Role: FOLLOWER
I20260808 20:58:03.121153 393 consensus_queue.cc:260] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96db77d5feb4af19960b94a877229b5" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37141 } }
I20260808 20:58:03.121243 393 raft_consensus.cc:388] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:58:03.121287 393 raft_consensus.cc:482] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:58:03.121342 393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 20:58:03 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:03.122079 393 raft_consensus.cc:504] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96db77d5feb4af19960b94a877229b5" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37141 } }
I20260808 20:58:03.122195 393 leader_election.cc:302] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c96db77d5feb4af19960b94a877229b5; no voters:
I20260808 20:58:03.122429 393 leader_election.cc:288] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:58:03.122605 401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:58:03.122732 393 sys_catalog.cc:564] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:58:03.122856 401 raft_consensus.cc:686] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [term 1 LEADER]: Becoming Leader. State: Replica: c96db77d5feb4af19960b94a877229b5, State: Running, Role: LEADER
I20260808 20:58:03.123030 401 consensus_queue.cc:237] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96db77d5feb4af19960b94a877229b5" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37141 } }
I20260808 20:58:03.123545 401 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c96db77d5feb4af19960b94a877229b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96db77d5feb4af19960b94a877229b5" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37141 } } }
I20260808 20:58:03.123719 401 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [sys.catalog]: This master's current role is: LEADER
I20260808 20:58:03.123939 401 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c96db77d5feb4af19960b94a877229b5. Latest consensus state: current_term: 1 leader_uuid: "c96db77d5feb4af19960b94a877229b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96db77d5feb4af19960b94a877229b5" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37141 } } }
I20260808 20:58:03.124080 401 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5 [sys.catalog]: This master's current role is: LEADER
I20260808 20:58:03.124274 418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:58:03.124557 393 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 20:58:03.129233 418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:58:03.130686 397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:03.115328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.15.232.190:37141)
0808 20:58:03.115552 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:03.115563 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:03.115597 (+ 34us) server_negotiation.cc:407] Connection header received
0808 20:58:03.116149 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:03.116157 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:03.116209 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:03.116367 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:03.117161 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:03.118362 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:03.119138 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:03.119646 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:03.119777 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:03.122468 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:03.122473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:03.122473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:03.122488 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:03.124927 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:03.125445 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:03.125449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:03.125449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:03.125544 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:03.127232 (+ 1688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:03.127265 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:03.127265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:03.128655 (+ 1390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:03.129695 (+ 1040us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:03.129827 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 20:58:03.130362 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":111,"threads_started":1}
I20260808 20:58:03.137125 418 catalog_manager.cc:1347] Generated new cluster ID: f596ad05acd043fc8f2df431ec819c81
I20260808 20:58:03.137271 418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:58:03.154213 418 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:58:03.155325 418 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:58:03.168054 418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5: Generated new TSK 0
I20260808 20:58:03.168321 418 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:58:03.931942 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:37977/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37141
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:58:04.034853 455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:04.035092 455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:04.035120 455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:04.036577 455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:04.036630 455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:04.036660 455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:58:04.038255 455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.232.148:37977/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:04.038534 455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:04.038792 455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:04.044255 467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:04.044301 463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:04.044466 455 server_base.cc:1034] running on GCE node
W20260808 20:58:04.044302 465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:04.044868 455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:04.045154 455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:04.046330 455 hybrid_clock.cc:648] HybridClock initialized: now 1786222684046314 us; error 34 us; skew 500 ppm
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:04.049085 455 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:58:04.049568 455 webserver.cc:460] Webserver started at http://127.15.232.129:41183/ using document root <none> and password file <none>
I20260808 20:58:04.049817 455 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:04.049888 455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:04.050060 455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:04.065397 455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "ff888a98b44d40cea15153266a07676a"
format_stamp: "Formatted at 2026-08-08 20:58:04 on dist-test-slave-c12x"
server_key: "0b58ea9194f225a16444f61a719aacca"
server_key_iv: "11cfb7716a663d7e7a822e70a86c7187"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:04.066143 455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "ff888a98b44d40cea15153266a07676a"
format_stamp: "Formatted at 2026-08-08 20:58:04 on dist-test-slave-c12x"
server_key: "0b58ea9194f225a16444f61a719aacca"
server_key_iv: "11cfb7716a663d7e7a822e70a86c7187"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:04.071838 455 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.002s
I20260808 20:58:04.087613 475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:04.087886 455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 20:58:04.087999 455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "ff888a98b44d40cea15153266a07676a"
format_stamp: "Formatted at 2026-08-08 20:58:04 on dist-test-slave-c12x"
server_key: "0b58ea9194f225a16444f61a719aacca"
server_key_iv: "11cfb7716a663d7e7a822e70a86c7187"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:04.088128 455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:04.098067 455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:04.103282 455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:04.103538 455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:04.103946 455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:58:04.104391 455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:58:04.104461 455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:04.104538 455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:58:04.104573 455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:04.112691 455 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:45233
I20260808 20:58:04.113257 455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:58:04.119359 630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:45233 every 8 connection(s)
I20260808 20:58:04.119695 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 455
I20260808 20:58:04.119819 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:58:04.120033 16290 external_mini_cluster.cc:1428] Setting key 2172c0bbbed80f8b4e6edc305bb086e0
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
I20260808 20:58:04.135568 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.123094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:51363 (local address 127.15.232.190:37141)
0808 20:58:04.125834 (+ 2740us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.125836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.125844 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:58:04.125869 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.125869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.125881 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.125997 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:04.126668 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.127201 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.128327 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.128593 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.128662 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.131491 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:04.131493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:04.131494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:04.131502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:04.133723 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:04.134292 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:04.134293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:04.134294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:04.134439 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:04.134774 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:04.134775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:04.134776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:04.134908 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:04.134993 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.135309 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.135409 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2705,"thread_start_us":120,"threads_started":1}
I20260808 20:58:04.135703 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.123086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37141 (local address 127.15.232.129:51363)
0808 20:58:04.123302 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:04.123347 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:04.124248 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:04.126101 (+ 1853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:04.126105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:04.126334 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:04.126553 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.126558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.127315 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.127316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.128219 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.128221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.128737 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.128738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.128887 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.129575 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:04.129586 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:04.131363 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:04.133835 (+ 2472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:04.133843 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:04.133846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:04.134165 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:04.134520 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:04.134521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:04.134521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:04.134674 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:04.134996 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:04.135004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:04.135161 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 20:58:04.135272 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 20:58:04.135434 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260808 20:58:04.136356 631 heartbeater.cc:347] Connected to a master server at 127.15.232.190:37141
I20260808 20:58:04.136472 631 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:04.136690 631 heartbeater.cc:511] Master 127.15.232.190:37141 requested a full tablet report, sending...
I20260808 20:58:04.137331 347 ts_manager.cc:194] Registered new tserver with Master: ff888a98b44d40cea15153266a07676a (127.15.232.129:45233)
I20260808 20:58:04.138229 347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:51363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:58:04.155689 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:37977/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37141
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:58:04.278620 641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:04.278923 641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:04.278960 641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:04.280773 641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:04.280840 641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:04.280884 641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:58:04.283288 641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.232.148:37977/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:04.283583 641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:04.283847 641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:04.290923 654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:04.291142 650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:04.291262 649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:04.292205 641 server_base.cc:1034] running on GCE node
I20260808 20:58:04.292451 641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:04.292766 641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:04.293958 641 hybrid_clock.cc:648] HybridClock initialized: now 1786222684293918 us; error 54 us; skew 500 ppm
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:04.297848 641 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:58:04.298434 641 webserver.cc:460] Webserver started at http://127.15.232.130:34649/ using document root <none> and password file <none>
I20260808 20:58:04.298738 641 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:04.298808 641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:04.298942 641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:04.314518 641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "36c621fe0c6642bb9bd48af63172c50f"
format_stamp: "Formatted at 2026-08-08 20:58:04 on dist-test-slave-c12x"
server_key: "7a533e57220c548817e8f498cc269978"
server_key_iv: "e799ad31a9fb5da8a8ac280b840fdc91"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:04.315080 641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "36c621fe0c6642bb9bd48af63172c50f"
format_stamp: "Formatted at 2026-08-08 20:58:04 on dist-test-slave-c12x"
server_key: "7a533e57220c548817e8f498cc269978"
server_key_iv: "e799ad31a9fb5da8a8ac280b840fdc91"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:04.317312 641 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 20:58:04.339399 664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:04.339705 641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:58:04.339805 641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "36c621fe0c6642bb9bd48af63172c50f"
format_stamp: "Formatted at 2026-08-08 20:58:04 on dist-test-slave-c12x"
server_key: "7a533e57220c548817e8f498cc269978"
server_key_iv: "e799ad31a9fb5da8a8ac280b840fdc91"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:04.339901 641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:04.360200 641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:04.362912 641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:04.363068 641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:04.363323 641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:58:04.363749 641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:58:04.363793 641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:04.363829 641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:58:04.363886 641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:04.372928 641 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:43643
I20260808 20:58:04.373471 641 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:58:04.374109 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 641
I20260808 20:58:04.373991 818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:43643 every 8 connection(s)
I20260808 20:58:04.374228 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:58:04.374464 16290 external_mini_cluster.cc:1428] Setting key 5079147d08267ea23dc2deb2e60cb352
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:04.385483 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.374472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37141 (local address 127.15.232.130:55283)
0808 20:58:04.374686 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:04.374740 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:04.375379 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:04.375686 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:04.375691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:04.375936 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:04.376212 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.376217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.377254 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.377255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.378170 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.378173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.378721 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.378723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.378882 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.379555 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:04.379566 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:04.381345 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:04.383593 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:04.383602 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:04.383605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:04.383948 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:04.384197 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:04.384198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:04.384198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:04.384352 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:04.384688 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:04.384696 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:04.384881 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 20:58:04.384999 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 20:58:04.385188 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260808 20:58:04.386070 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.374524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:55283 (local address 127.15.232.190:37141)
0808 20:58:04.374612 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.374615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.374841 (+ 226us) server_negotiation.cc:407] Connection header received
0808 20:58:04.375519 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.375521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.375552 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.375614 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:04.376317 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.377103 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.378273 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.378612 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.378688 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.381448 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:04.381450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:04.381450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:04.381459 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:04.383485 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:04.384045 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:04.384047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:04.384047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:04.384123 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:04.384431 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:04.384432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:04.384432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:04.384588 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:04.384656 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.385864 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.385972 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
I20260808 20:58:04.386899 819 heartbeater.cc:347] Connected to a master server at 127.15.232.190:37141
I20260808 20:58:04.387003 819 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:04.387228 819 heartbeater.cc:511] Master 127.15.232.190:37141 requested a full tablet report, sending...
I20260808 20:58:04.387703 347 ts_manager.cc:194] Registered new tserver with Master: 36c621fe0c6642bb9bd48af63172c50f (127.15.232.130:43643)
I20260808 20:58:04.388085 347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:55283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:58:04.411273 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:37977/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37141
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:58:04.548749 828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:04.549116 828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:04.549170 828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:04.551724 828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:04.551795 828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:04.551841 828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:58:04.554477 828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.232.148:37977/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:37141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:04.554811 828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:04.555193 828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:04.562790 842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:04.562793 839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:04.563328 837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:04.563730 828 server_base.cc:1034] running on GCE node
I20260808 20:58:04.563974 828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:04.564325 828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:04.565531 828 hybrid_clock.cc:648] HybridClock initialized: now 1786222684565505 us; error 44 us; skew 500 ppm
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:04.569288 828 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:58:04.570052 828 webserver.cc:460] Webserver started at http://127.15.232.131:34019/ using document root <none> and password file <none>
I20260808 20:58:04.570389 828 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:04.570461 828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:04.570670 828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:04.589699 828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "9fed96b9e5604161bc0ec13837430e28"
format_stamp: "Formatted at 2026-08-08 20:58:04 on dist-test-slave-c12x"
server_key: "78e388576f1ad7f79e595809d8695bec"
server_key_iv: "cff03c0125df5f8e8a43140e3ae31e3a"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:04.600029 828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "9fed96b9e5604161bc0ec13837430e28"
format_stamp: "Formatted at 2026-08-08 20:58:04 on dist-test-slave-c12x"
server_key: "78e388576f1ad7f79e595809d8695bec"
server_key_iv: "cff03c0125df5f8e8a43140e3ae31e3a"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:04.604712 828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260808 20:58:04.625710 849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:04.626015 828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 20:58:04.626143 828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "9fed96b9e5604161bc0ec13837430e28"
format_stamp: "Formatted at 2026-08-08 20:58:04 on dist-test-slave-c12x"
server_key: "78e388576f1ad7f79e595809d8695bec"
server_key_iv: "cff03c0125df5f8e8a43140e3ae31e3a"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:04.626245 828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:04.639324 828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:04.642130 828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:04.642339 828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:04.642684 828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:58:04.643121 828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:58:04.643174 828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:04.643208 828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:58:04.643227 828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:04.659240 828 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:40149
I20260808 20:58:04.659291 998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:40149 every 8 connection(s)
I20260808 20:58:04.660005 828 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:58:04.662344 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 828
I20260808 20:58:04.662482 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:58:04.662738 16290 external_mini_cluster.cc:1428] Setting key 52c9a27d4530fdddb4737223f24371c6
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:04.673095 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.661020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:49811 (local address 127.15.232.190:37141)
0808 20:58:04.661073 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.661076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.661387 (+ 311us) server_negotiation.cc:407] Connection header received
0808 20:58:04.662032 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.662034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.662048 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.662102 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:04.662712 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.663492 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.665574 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.666042 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.666157 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.669566 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:04.669568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:04.669569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:04.669579 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:04.671303 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:04.671874 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:04.671875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:04.671876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:04.671954 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:04.672281 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:04.672282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:04.672282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:04.672444 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:04.672536 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.672883 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.672986 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 20:58:04.673454 1001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.661043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37141 (local address 127.15.232.131:49811)
0808 20:58:04.661238 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:04.661282 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:04.661896 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:04.662192 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:04.662196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:04.662400 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:04.662611 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.662616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.663586 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.663587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.665312 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.665319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.666374 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.666375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.666537 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.667526 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:04.667538 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:04.669425 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:04.671415 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:04.671422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:04.671425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:04.671766 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:04.672055 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:04.672056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:04.672056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:04.672200 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:04.672528 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:04.672533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:04.672717 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 20:58:04.672835 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 20:58:04.673016 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":114,"threads_started":1}
I20260808 20:58:04.675644 999 heartbeater.cc:347] Connected to a master server at 127.15.232.190:37141
I20260808 20:58:04.675956 999 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:04.676383 999 heartbeater.cc:511] Master 127.15.232.190:37141 requested a full tablet report, sending...
I20260808 20:58:04.677035 347 ts_manager.cc:194] Registered new tserver with Master: 9fed96b9e5604161bc0ec13837430e28 (127.15.232.131:40149)
I20260808 20:58:04.677539 347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:49811
I20260808 20:58:04.686278 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:58:04.696686 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.688371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51924 (local address 127.15.232.190:37141)
0808 20:58:04.688419 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.688421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.688553 (+ 132us) server_negotiation.cc:407] Connection header received
0808 20:58:04.688609 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.688610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.688622 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.688690 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:04.689485 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.690289 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.691346 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.691672 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.691741 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.693523 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:04.693525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:04.693525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:04.693534 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:04.695300 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:04.695770 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:04.695771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:04.695771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:04.695839 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:04.696076 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:04.696077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:04.696077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:04.696235 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:04.696337 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.696491 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.696583 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:58:04.698477 347 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:58:04.700902 347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:58:04.711725 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.704588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:45233 (local address 127.0.0.1:55902)
0808 20:58:04.706212 (+ 1624us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:04.706256 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:04.706322 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:04.706687 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:04.706691 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:04.706694 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:04.706771 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.706773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.708479 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.708480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.709843 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.709846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.711316 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.711317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.711406 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.711421 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 20:58:04.711556 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 20:58:04.711629 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1592,"thread_start_us":87,"threads_started":1}
I20260808 20:58:04.715011 1019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.704611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:43643 (local address 127.0.0.1:59316)
0808 20:58:04.706772 (+ 2161us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:04.706783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:04.706830 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:04.707210 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:04.707211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:04.707213 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:04.707259 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.707260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.711546 (+ 4286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.711548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.713523 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.713526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.714735 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.714736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.714804 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.714809 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:58:04.714865 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 20:58:04.714911 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2142,"thread_start_us":149,"threads_started":1}
I20260808 20:58:04.715646 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.705241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59316 (local address 127.15.232.130:43643)
0808 20:58:04.706957 (+ 1716us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.706962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.706971 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:58:04.707033 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.707035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.707050 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.707123 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:04.707346 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.708484 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.713653 (+ 5169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.714487 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.714618 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.714692 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.715497 (+ 805us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.715559 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1675,"thread_start_us":195,"threads_started":1}
I20260808 20:58:04.715920 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.704868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55902 (local address 127.15.232.129:45233)
0808 20:58:04.706487 (+ 1619us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.706492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.706505 (+ 13us) server_negotiation.cc:407] Connection header received
0808 20:58:04.706545 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.706547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.706564 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.706664 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:04.706865 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.708375 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.709954 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.711116 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.711197 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.711268 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.715337 (+ 4069us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.715398 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1580,"thread_start_us":65,"threads_started":1}
I20260808 20:58:04.717921 539 tablet_service.cc:1467] Processing CreateTablet for tablet ebe83b65e68948b2888c0c5ac4d913c4 (DEFAULT_TABLE table=test-table [id=03bf7f9357ff4e39b37b882bb5e94e8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:58:04.718382 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebe83b65e68948b2888c0c5ac4d913c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:04.718393 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.704865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:40149 (local address 127.0.0.1:46334)
0808 20:58:04.708585 (+ 3720us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:04.708596 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:04.708695 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:04.709268 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:04.709269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:04.709272 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:04.709315 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.709316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.710576 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.710577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.712585 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.712588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.718093 (+ 5505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.718094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.718174 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.718179 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:58:04.718232 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 20:58:04.718285 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3698,"thread_start_us":136,"threads_started":1}
I20260808 20:58:04.718712 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.705545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.15.232.131:40149)
0808 20:58:04.706137 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.706144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.708663 (+ 2519us) server_negotiation.cc:407] Connection header received
0808 20:58:04.708789 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.708792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.708809 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.708865 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:04.709399 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.710468 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.712732 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.717571 (+ 4839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.717655 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.717717 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.718572 (+ 855us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.718629 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":172,"threads_started":1}
I20260808 20:58:04.719859 911 tablet_service.cc:1467] Processing CreateTablet for tablet ebe83b65e68948b2888c0c5ac4d913c4 (DEFAULT_TABLE table=test-table [id=03bf7f9357ff4e39b37b882bb5e94e8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:58:04.720299 911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebe83b65e68948b2888c0c5ac4d913c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:04.724530 721 tablet_service.cc:1467] Processing CreateTablet for tablet ebe83b65e68948b2888c0c5ac4d913c4 (DEFAULT_TABLE table=test-table [id=03bf7f9357ff4e39b37b882bb5e94e8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:58:04.724961 721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebe83b65e68948b2888c0c5ac4d913c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:04.726131 1025 tablet_bootstrap.cc:492] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: Bootstrap starting.
I20260808 20:58:04.727656 1025 tablet_bootstrap.cc:654] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:04.728009 1025 log.cc:824] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:04.728909 1025 tablet_bootstrap.cc:492] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: No bootstrap required, opened a new log
I20260808 20:58:04.728976 1025 ts_tablet_manager.cc:1397] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 20:58:04.731254 1028 tablet_bootstrap.cc:492] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: Bootstrap starting.
I20260808 20:58:04.731302 1025 raft_consensus.cc:348] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } }
I20260808 20:58:04.731523 1025 raft_consensus.cc:374] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:04.731573 1025 raft_consensus.cc:729] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fed96b9e5604161bc0ec13837430e28, State: Initialized, Role: FOLLOWER
I20260808 20:58:04.732894 1028 tablet_bootstrap.cc:654] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:04.733187 1022 tablet_bootstrap.cc:492] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a: Bootstrap starting.
I20260808 20:58:04.734052 1022 tablet_bootstrap.cc:654] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:04.734484 1022 log.cc:824] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:04.734581 1025 consensus_queue.cc:260] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } }
I20260808 20:58:04.734942 1025 ts_tablet_manager.cc:1428] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260808 20:58:04.735090 999 heartbeater.cc:503] Master 127.15.232.190:37141 was elected leader, sending a full tablet report...
I20260808 20:58:04.735263 1028 log.cc:824] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:04.736894 1022 tablet_bootstrap.cc:492] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a: No bootstrap required, opened a new log
I20260808 20:58:04.737166 1022 ts_tablet_manager.cc:1397] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 20:58:04.740185 1022 raft_consensus.cc:348] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } }
I20260808 20:58:04.740412 1022 raft_consensus.cc:374] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:04.740461 1022 raft_consensus.cc:729] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff888a98b44d40cea15153266a07676a, State: Initialized, Role: FOLLOWER
I20260808 20:58:04.741082 1028 tablet_bootstrap.cc:492] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: No bootstrap required, opened a new log
I20260808 20:58:04.741182 1028 ts_tablet_manager.cc:1397] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260808 20:58:04.743602 1022 consensus_queue.cc:260] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } }
I20260808 20:58:04.743654 1028 raft_consensus.cc:348] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } }
I20260808 20:58:04.743863 1028 raft_consensus.cc:374] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:04.743912 1028 raft_consensus.cc:729] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c621fe0c6642bb9bd48af63172c50f, State: Initialized, Role: FOLLOWER
I20260808 20:58:04.744091 1028 consensus_queue.cc:260] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } }
I20260808 20:58:04.744288 1022 ts_tablet_manager.cc:1428] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260808 20:58:04.744472 631 heartbeater.cc:503] Master 127.15.232.190:37141 was elected leader, sending a full tablet report...
I20260808 20:58:04.744516 1028 ts_tablet_manager.cc:1428] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 20:58:04.744637 819 heartbeater.cc:503] Master 127.15.232.190:37141 was elected leader, sending a full tablet report...
I20260808 20:58:04.802003 1038 raft_consensus.cc:482] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:58:04.802163 1038 raft_consensus.cc:504] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } }
I20260808 20:58:04.803843 1038 leader_election.cc:288] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff888a98b44d40cea15153266a07676a (127.15.232.129:45233), 9fed96b9e5604161bc0ec13837430e28 (127.15.232.131:40149)
I20260808 20:58:04.806565 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.802743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:45233 (local address 127.15.232.130:38847)
0808 20:58:04.802853 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:04.802864 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:04.802905 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:04.803173 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:04.803173 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:04.803175 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:04.803263 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.803265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.804236 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.804237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.805444 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.805447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.806301 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.806302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.806380 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.806383 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:58:04.806428 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 20:58:04.806470 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":7}
I20260808 20:58:04.806851 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.802982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:38847 (local address 127.15.232.129:45233)
0808 20:58:04.803027 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.803029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.803037 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:58:04.803064 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.803064 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.803075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.803122 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:04.803337 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.804132 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.805590 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.806191 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.806259 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.806269 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.806740 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.806787 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:58:04.807124 577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe83b65e68948b2888c0c5ac4d913c4" candidate_uuid: "36c621fe0c6642bb9bd48af63172c50f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff888a98b44d40cea15153266a07676a" is_pre_election: true
I20260808 20:58:04.807303 577 raft_consensus.cc:2454] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36c621fe0c6642bb9bd48af63172c50f in term 0.
I20260808 20:58:04.807862 665 leader_election.cc:302] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c621fe0c6642bb9bd48af63172c50f, ff888a98b44d40cea15153266a07676a; no voters:
I20260808 20:58:04.808257 1038 raft_consensus.cc:2781] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:58:04.808336 1038 raft_consensus.cc:482] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:58:04.808365 1038 raft_consensus.cc:3037] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:04.808991 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.804208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:51591 (local address 127.15.232.131:40149)
0808 20:58:04.804254 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.804256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.804382 (+ 126us) server_negotiation.cc:407] Connection header received
0808 20:58:04.804485 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.804486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.804498 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.804528 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:04.804766 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.805530 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.807400 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.807970 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.808037 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.808047 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.808872 (+ 825us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.808922 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:58:04.809381 1039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.803681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:40149 (local address 127.15.232.130:51591)
0808 20:58:04.804319 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:04.804331 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:04.804408 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:04.804601 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:04.804602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:04.804604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:04.804688 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.804689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.805660 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.805661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.807173 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:04.807177 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.808701 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.808702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:04.808781 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.808786 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:58:04.809232 (+ 446us) client_negotiation.cc:240] Negotiation successful
0808 20:58:04.809281 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":119,"threads_started":1}
I20260808 20:58:04.811242 940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe83b65e68948b2888c0c5ac4d913c4" candidate_uuid: "36c621fe0c6642bb9bd48af63172c50f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fed96b9e5604161bc0ec13837430e28" is_pre_election: true
I20260808 20:58:04.811409 940 raft_consensus.cc:2454] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36c621fe0c6642bb9bd48af63172c50f in term 0.
I20260808 20:58:04.809440 1038 raft_consensus.cc:504] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } }
I20260808 20:58:04.811906 1038 leader_election.cc:288] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [CANDIDATE]: Term 1 election: Requested vote from peers ff888a98b44d40cea15153266a07676a (127.15.232.129:45233), 9fed96b9e5604161bc0ec13837430e28 (127.15.232.131:40149)
I20260808 20:58:04.812275 940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe83b65e68948b2888c0c5ac4d913c4" candidate_uuid: "36c621fe0c6642bb9bd48af63172c50f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fed96b9e5604161bc0ec13837430e28"
I20260808 20:58:04.812355 940 raft_consensus.cc:3037] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:04.813457 940 raft_consensus.cc:2454] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36c621fe0c6642bb9bd48af63172c50f in term 1.
I20260808 20:58:04.813719 666 leader_election.cc:302] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c621fe0c6642bb9bd48af63172c50f, 9fed96b9e5604161bc0ec13837430e28; no voters:
I20260808 20:58:04.813808 1038 raft_consensus.cc:2781] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:58:04.813977 1038 raft_consensus.cc:686] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 1 LEADER]: Becoming Leader. State: Replica: 36c621fe0c6642bb9bd48af63172c50f, State: Running, Role: LEADER
I20260808 20:58:04.814078 1038 consensus_queue.cc:237] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } }
I20260808 20:58:04.815064 347 catalog_manager.cc:5495] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f reported cstate change: term changed from 0 to 1, leader changed from <none> to 36c621fe0c6642bb9bd48af63172c50f (127.15.232.130). New cstate: current_term: 1 leader_uuid: "36c621fe0c6642bb9bd48af63172c50f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c621fe0c6642bb9bd48af63172c50f" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 } health_report { overall_health: UNKNOWN } } }
I20260808 20:58:04.815531 577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebe83b65e68948b2888c0c5ac4d913c4" candidate_uuid: "36c621fe0c6642bb9bd48af63172c50f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff888a98b44d40cea15153266a07676a"
I20260808 20:58:04.815629 577 raft_consensus.cc:3037] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:04.816587 577 raft_consensus.cc:2454] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36c621fe0c6642bb9bd48af63172c50f in term 1.
Aug 08 20:58:04 dist-test-slave-c12x krb5kdc[32131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222684, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:04.838753 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.835126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.15.232.130:43643)
0808 20:58:04.835173 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:04.835175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:04.835187 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:58:04.835288 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:04.835289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:04.835314 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:04.835412 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:58:04.835630 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.836387 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.837518 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:04.837819 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:04.837890 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:04.838136 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:58:04.838285 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:58:04.838366 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:04.838643 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 20:58:04.838667 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:58:04.842137 940 raft_consensus.cc:1261] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [term 1 FOLLOWER]: Refusing update from remote peer 36c621fe0c6642bb9bd48af63172c50f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:58:04.842559 1040 consensus_queue.cc:1035] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fed96b9e5604161bc0ec13837430e28" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:58:04.851991 1044 mvcc.cc:205] Tried to move back new op lower bound from 7316368117109821440 to 7316368116999245824. Current Snapshot: MvccSnapshot[applied={T|T < 7316368117109821440}]
I20260808 20:58:04.853508 1043 mvcc.cc:205] Tried to move back new op lower bound from 7316368117109821440 to 7316368116999245824. Current Snapshot: MvccSnapshot[applied={T|T < 7316368117109821440}]
I20260808 20:58:04.853901 577 raft_consensus.cc:1261] T ebe83b65e68948b2888c0c5ac4d913c4 P ff888a98b44d40cea15153266a07676a [term 1 FOLLOWER]: Refusing update from remote peer 36c621fe0c6642bb9bd48af63172c50f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:58:04.854241 1038 consensus_queue.cc:1035] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff888a98b44d40cea15153266a07676a" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:58:04.856906 347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:58:04.857008 347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:58:04.857988 347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c96db77d5feb4af19960b94a877229b5: Sending DeleteTablet for 3 replicas of tablet ebe83b65e68948b2888c0c5ac4d913c4
I20260808 20:58:04.858381 539 tablet_service.cc:1514] Processing DeleteTablet for tablet ebe83b65e68948b2888c0c5ac4d913c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:58:04 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:55902
I20260808 20:58:04.858714 721 tablet_service.cc:1514] Processing DeleteTablet for tablet ebe83b65e68948b2888c0c5ac4d913c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:58:04 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:59316
I20260808 20:58:04.858743 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 455
I20260808 20:58:04.859185 911 tablet_service.cc:1514] Processing DeleteTablet for tablet ebe83b65e68948b2888c0c5ac4d913c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:58:04 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:46334
I20260808 20:58:04.868646 1053 tablet_replica.cc:330] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: stopping tablet replica
I20260808 20:58:04.868883 1053 raft_consensus.cc:2229] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:58:04.869077 1053 raft_consensus.cc:2258] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:58:04.869374 1054 tablet_replica.cc:330] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: stopping tablet replica
I20260808 20:58:04.869557 1054 raft_consensus.cc:2229] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:58:04.869731 1054 raft_consensus.cc:2258] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:58:04.870630 1053 ts_tablet_manager.cc:1905] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:58:04.871125 1054 ts_tablet_manager.cc:1905] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:58:04.874083 1053 ts_tablet_manager.cc:1918] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:58:04.874177 1053 log.cc:1196] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/ebe83b65e68948b2888c0c5ac4d913c4
I20260808 20:58:04.874634 1054 ts_tablet_manager.cc:1918] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:58:04.874950 1054 log.cc:1196] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/ebe83b65e68948b2888c0c5ac4d913c4
I20260808 20:58:04.875689 1054 ts_tablet_manager.cc:1939] T ebe83b65e68948b2888c0c5ac4d913c4 P 9fed96b9e5604161bc0ec13837430e28: Deleting consensus metadata
I20260808 20:58:04.878023 1053 ts_tablet_manager.cc:1939] T ebe83b65e68948b2888c0c5ac4d913c4 P 36c621fe0c6642bb9bd48af63172c50f: Deleting consensus metadata
I20260808 20:58:04.878580 326 catalog_manager.cc:4841] TS 36c621fe0c6642bb9bd48af63172c50f (127.15.232.130:43643): tablet ebe83b65e68948b2888c0c5ac4d913c4 (table test-table [id=03bf7f9357ff4e39b37b882bb5e94e8c]) successfully deleted
I20260808 20:58:04.878706 326 catalog_manager.cc:4841] TS 9fed96b9e5604161bc0ec13837430e28 (127.15.232.131:40149): tablet ebe83b65e68948b2888c0c5ac4d913c4 (table test-table [id=03bf7f9357ff4e39b37b882bb5e94e8c]) successfully deleted
W20260808 20:58:04.881331 665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:45233 (error 108)
W20260808 20:58:04.881891 325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:45233 (error 108)
I20260808 20:58:04.882871 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:04.882698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:45233 (local address 127.0.0.1:55910)
0808 20:58:04.882774 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:04.882829 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 20:58:04.882858 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 641
W20260808 20:58:04.889838 325 catalog_manager.cc:4584] TS ff888a98b44d40cea15153266a07676a (127.15.232.129:45233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ebe83b65e68948b2888c0c5ac4d913c4: Network error: Client connection negotiation failed: client connection to 127.15.232.129:45233: connect: Connection refused (error 111)
I20260808 20:58:04.898650 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 828
I20260808 20:58:04.936267 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 32763
I20260808 20:58:04.954758 16290 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 20:58:05.311757 16290 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 20:58:05.311887 16290 mini_ranger.cc:66] Stopping Ranger...
I20260808 20:58:05.668125 16290 mini_ranger.cc:68] Stopped Ranger
2026-08-08 20:58:05.668 UTC [32171] LOG: received smart shutdown request
2026-08-08 20:58:05.673 UTC [32171] LOG: background worker "logical replication launcher" (PID 32184) exited with exit code 1
2026-08-08 20:58:05.673 UTC [32176] LOG: shutting down
2026-08-08 20:58:05.889 UTC [32171] LOG: database system is shut down
2026-08-08T20:58:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51222 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:58:06 dist-test-slave-c12x krb5kdc[1067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:58:06 dist-test-slave-c12x krb5kdc[1067](info): set up 2 sockets
Aug 08 20:58:06 dist-test-slave-c12x krb5kdc[1067](info): commencing operation
krb5kdc: starting...
W20260808 20:58:07.283210 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:58:07 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:58:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:58:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.232.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.232.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/rangeradmin_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/rangeradmin_127.15.232.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.232.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.232.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/rangerlookup_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/rangerlookup_127.15.232.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.232.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.232.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/HTTP_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/HTTP_127.15.232.148@KRBTEST.COM.keytab.
I20260808 20:58:07.475766 16290 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0tXHI2/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/postgres -l logfile start
2026-08-08 20:58:10.409 UTC [1111] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 20:58:10.409 UTC [1111] LOG: listening on IPv4 address "127.15.232.148", port 39349
2026-08-08 20:58:10.414 UTC [1111] LOG: listening on Unix socket "/tmp/.s.PGSQL.39349"
2026-08-08 20:58:10.440 UTC [1115] LOG: database system was shut down at 2026-08-08 20:58:08 UTC
2026-08-08 20:58:10.445 UTC [1111] LOG: database system is ready to accept connections
I20260808 20:58:11.612891 16290 mini_postgres.cc:96] Postgres bound to 39349
2026-08-08 20:58:11.622 UTC [1127] FATAL: database "slave" does not exist
127.15.232.148:39349 - accepting connections
I20260808 20:58:11.624159 16290 mini_ranger.cc:161] Starting Ranger...
I20260808 20:58:11.642158 16290 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 20:58:12.251787 16290 mini_ranger.cc:87] Created ranger Postgres database
I20260808 20:58:12.251915 16290 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-admin
2026-08-08 20:58:12,562 [I] DB FLAVOR :POSTGRES
2026-08-08 20:58:12,562 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 20:58:12,563 [I] Checking connection..
2026-08-08 20:58:12,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 20:58:12,835 [I] Checking connection passed.
2026-08-08 20:58:12,835 [I] --------- Verifying version history table ---------
2026-08-08 20:58:12,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 20:58:13,132 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 20:58:13,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 20:58:13,407 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 20:58:13,408 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 20:58:13,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 20:58:13.682 UTC [1211] WARNING: there is no transaction in progress
2026-08-08 20:58:13,693 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 20:58:13,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 20:58:14,002 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 20:58:14,003 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 20:58:14,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 20:58:14,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 20:58:14,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-c12x.c.gcp-upstream.internal','N') ;"
2026-08-08 20:58:14,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 20:58:15,073 [I] Table x_portal_user does not exist in database ranger
2026-08-08 20:58:15,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 20:58:15,375 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 20:58:15,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 20:58:15,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 20:58:15,901 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 20:58:15,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 20:58:16.978 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:16.995 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.014 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.032 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.054 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.079 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.151 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.160 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.176 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.187 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.199 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.215 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.226 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.233 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.241 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17.249 UTC [1370] WARNING: there is no transaction in progress
2026-08-08 20:58:17,987 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 20:58:17,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 20:58:18,278 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 20:58:18,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 20:58:18,541 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 20:58:18,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 20:58:18,800 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 20:58:18,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 20:58:19,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 20:58:19,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-c12x.c.gcp-upstream.internal';"
2026-08-08 20:58:19,590 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 20:58:19,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 20:58:19,866 [I] DB_PATCHES have already been applied
I20260808 20:58:19.872813 16290 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-admin:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0tXHI2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 20:58:19.872895 16290 mini_ranger.cc:193] Using host: 127.15.232.148
I20260808 20:58:19.877584 16290 mini_ranger.cc:227] Ranger admin URL: http://127.15.232.148:44287
Aug 08, 2026 8:58:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 8:58:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 8:58:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 8:58:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 8:58:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 8:58:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 8:58:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 20:58:21.245352 16290 mini_ranger.cc:160] Time spent starting Ranger: real 9.621s user 0.002s sys 0.010s
Aug 08, 2026 8:58:42 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 8:58:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 8:58:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 8:58:42 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 8:58:43 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 20:58:43.798306 16290 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.232.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.232.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/rangerkms_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.232.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/rangerkms_127.15.232.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.232.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.232.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.232.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/HTTP_127.15.232.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.232.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/HTTP_127.15.232.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 20:58:43 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222723, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 20:58:43.892493 16290 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 20:58:43.904110 16290 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 20:58:44.862870 16290 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 20:58:44.862967 16290 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-kms
I20260808 20:58:44.862989 16290 mini_ranger_kms.cc:227] Using postgres at 127.15.232.148:39349
2026-08-08 20:58:45,058 [I] DB FLAVOR :POSTGRES
2026-08-08 20:58:45,058 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 20:58:45,059 [I] Checking connection
2026-08-08 20:58:45,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 20:58:45,314 [I] connection success
2026-08-08 20:58:45,314 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 20:58:45,314 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 20:58:45,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 20:58:45,576 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 20:58:45,576 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 20:58:45,576 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 20:58:45,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.232.148:39349/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 20:58:45,877 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 20:58:46.336798 16290 mini_ranger_kms.cc:310] Created kms service
I20260808 20:58:46.568672 16290 mini_ranger_kms.cc:326] Created kudu user
I20260808 20:58:46.622490 16290 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 20:58:46 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222723, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:47.075235 16290 mini_ranger_kms.cc:384] Added ranger policy
I20260808 20:58:47.075484 16290 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-kms:/tmp/dist-test-task0tXHI2/build/release/bin/postgresql.jar:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0tXHI2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 20:58:47.075510 16290 mini_ranger_kms.cc:240] Using host: 127.15.232.148
I20260808 20:58:47.080454 16290 mini_ranger_kms.cc:276] Ranger KMS PID: 1625
I20260808 20:58:47.080576 16290 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.232.148:35927
Aug 08, 2026 8:58:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 8:58:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 8:58:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 8:58:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 8:58:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 8:58:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 20:58:49 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786222729, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:49.824338 16290 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.932s user 0.008s sys 0.168s
I20260808 20:58:49.824451 16290 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 20:58:49.824497 16290 mini_ranger_kms.cc:401] 127.15.232.148:35927/kms/v1/keys
Aug 08 20:58:50 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222729, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.148@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
Aug 08 20:58:50 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222729, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.232.148@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
Aug 08 20:58:52 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:58:52.686201 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36281
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:36281
--ranger_config_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:35927/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:58:52.777249 1698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:52.777475 1698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:52.777498 1698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:52.778652 1698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:58:52.778692 1698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:52.778707 1698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:52.778718 1698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:58:52.778730 1698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:58:52.780409 1698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.232.148:35927/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:36281
--ranger_config_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:52.780630 1698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:52.780807 1698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:52.786293 1707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:52.786301 1703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:52.786322 1698 server_base.cc:1034] running on GCE node
W20260808 20:58:52.786293 1704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:52.786787 1698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:52.787135 1698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:52.788286 1698 hybrid_clock.cc:648] HybridClock initialized: now 1786222732788260 us; error 40 us; skew 500 ppm
Aug 08 20:58:52 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222732, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:52.795285 1698 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:58:52.795871 1698 webserver.cc:460] Webserver started at http://127.15.232.190:35777/ using document root <none> and password file <none>
I20260808 20:58:52.796140 1698 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:52.796205 1698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:52.796367 1698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 20:58:52 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222732, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:52.852885 1698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "5338321e91fd4602acdc00d67a9a732d"
format_stamp: "Formatted at 2026-08-08 20:58:52 on dist-test-slave-c12x"
server_key: "0bcb45746d23a7097db26d58186c06f4"
server_key_iv: "cad3f84292c28e6df27e23201d89fc9e"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:52.853389 1698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "5338321e91fd4602acdc00d67a9a732d"
format_stamp: "Formatted at 2026-08-08 20:58:52 on dist-test-slave-c12x"
server_key: "0bcb45746d23a7097db26d58186c06f4"
server_key_iv: "cad3f84292c28e6df27e23201d89fc9e"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:52.855314 1698 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260808 20:58:52.872282 1716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:52.872609 1698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:58:52.872731 1698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "5338321e91fd4602acdc00d67a9a732d"
format_stamp: "Formatted at 2026-08-08 20:58:52 on dist-test-slave-c12x"
server_key: "0bcb45746d23a7097db26d58186c06f4"
server_key_iv: "cad3f84292c28e6df27e23201d89fc9e"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:52.872833 1698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:52.922041 1698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:52.923957 1698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:52.924126 1698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:52.928673 1698 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:36281
I20260808 20:58:52.928831 1792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:36281 every 8 connection(s)
I20260808 20:58:52.929167 1698 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:58:52.929946 1793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:52.932301 1793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Bootstrap starting.
I20260808 20:58:52.932927 1793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:52.933187 1793 log.cc:824] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:52.933969 1793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: No bootstrap required, opened a new log
I20260808 20:58:52.934782 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 1698
I20260808 20:58:52.934870 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:58:52.935043 16290 external_mini_cluster.cc:1428] Setting key 21e16f5e47098d235798477232462cde
I20260808 20:58:52.936383 1793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } }
I20260808 20:58:52.936549 1793 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:52.936590 1793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5338321e91fd4602acdc00d67a9a732d, State: Initialized, Role: FOLLOWER
I20260808 20:58:52.936766 1793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } }
I20260808 20:58:52.936887 1793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:58:52.936925 1793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:58:52.937008 1793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:52.938000 1793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } }
I20260808 20:58:52.938126 1793 leader_election.cc:302] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5338321e91fd4602acdc00d67a9a732d; no voters:
I20260808 20:58:52.938338 1793 leader_election.cc:288] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:58:52.938370 1801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:58:52.938475 1801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 1 LEADER]: Becoming Leader. State: Replica: 5338321e91fd4602acdc00d67a9a732d, State: Running, Role: LEADER
I20260808 20:58:52.938568 1801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } }
I20260808 20:58:52.938617 1793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:58:52.938884 1801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5338321e91fd4602acdc00d67a9a732d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } } }
I20260808 20:58:52.939052 1801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: This master's current role is: LEADER
I20260808 20:58:52.939256 1801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5338321e91fd4602acdc00d67a9a732d. Latest consensus state: current_term: 1 leader_uuid: "5338321e91fd4602acdc00d67a9a732d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } } }
I20260808 20:58:52.939386 1801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: This master's current role is: LEADER
I20260808 20:58:52.939409 1810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:58:52.940052 1810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:58:52.940323 1793 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 20:58:52.945632 1810 catalog_manager.cc:1347] Generated new cluster ID: 41265925d719411f8a91ddee82d38ff8
Aug 08 20:58:52 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:52.945793 1810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:58:52.952893 1799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:52.935619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35102 (local address 127.15.232.190:36281)
0808 20:58:52.935800 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:52.935807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:52.935867 (+ 60us) server_negotiation.cc:407] Connection header received
0808 20:58:52.936410 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:52.936419 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:52.936478 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:52.936840 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:52.937705 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:52.939154 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:52.940265 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:52.943039 (+ 2774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:52.943325 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:52.947907 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:52.947911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:52.947911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:52.947922 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:52.950774 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:52.951217 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:52.951220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:52.951221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:52.951339 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:52.951547 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:52.951548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:52.951548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:52.951895 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:52.952033 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:52.952480 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 20:58:52.952663 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260808 20:58:52.956014 1810 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:58:52.956604 1810 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:58:52.965296 1810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Generated new TSK 0
I20260808 20:58:52.965530 1810 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:58:53.692729 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:35927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36281
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:58:53.802657 1861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:53.802923 1861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:53.802968 1861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:53.804715 1861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:53.804800 1861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:53.804849 1861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:58:53.806633 1861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.232.148:35927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:53.806943 1861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:53.807232 1861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:53.812919 1870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:53.812790 1867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:53.812827 1873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:53.813390 1861 server_base.cc:1034] running on GCE node
I20260808 20:58:53.813606 1861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:53.813892 1861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:53.815091 1861 hybrid_clock.cc:648] HybridClock initialized: now 1786222733815067 us; error 35 us; skew 500 ppm
Aug 08 20:58:53 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222733, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:53.818262 1861 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:58:53.818764 1861 webserver.cc:460] Webserver started at http://127.15.232.129:45187/ using document root <none> and password file <none>
I20260808 20:58:53.819008 1861 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:53.819092 1861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:53.819253 1861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 20:58:53 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222733, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:53.834235 1861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "3c0561648e984cb59cd77e562785ca8f"
format_stamp: "Formatted at 2026-08-08 20:58:53 on dist-test-slave-c12x"
server_key: "1aba685b72d8b35d6ba1416ddc80f9d9"
server_key_iv: "756b9401ea46bb7fd8e0030161684b81"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:53.834815 1861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "3c0561648e984cb59cd77e562785ca8f"
format_stamp: "Formatted at 2026-08-08 20:58:53 on dist-test-slave-c12x"
server_key: "1aba685b72d8b35d6ba1416ddc80f9d9"
server_key_iv: "756b9401ea46bb7fd8e0030161684b81"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:53.839699 1861 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260808 20:58:53.859627 1883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:53.859918 1861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260808 20:58:53.860051 1861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "3c0561648e984cb59cd77e562785ca8f"
format_stamp: "Formatted at 2026-08-08 20:58:53 on dist-test-slave-c12x"
server_key: "1aba685b72d8b35d6ba1416ddc80f9d9"
server_key_iv: "756b9401ea46bb7fd8e0030161684b81"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:53.860172 1861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:53.877552 1861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:53.880366 1861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:53.880611 1861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:53.880926 1861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:58:53.881417 1861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:58:53.881508 1861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:53.881584 1861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:58:53.881635 1861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:53.889658 1861 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:32883
I20260808 20:58:53.890235 1861 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:58:53.890645 2049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:32883 every 8 connection(s)
I20260808 20:58:53.891520 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 1861
I20260808 20:58:53.891619 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:58:53.891793 16290 external_mini_cluster.cc:1428] Setting key 3090427158f29977418b6b47f6aad3f3
Aug 08 20:58:53 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222733, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:53.901634 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:53.891132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:42695 (local address 127.15.232.190:36281)
0808 20:58:53.892205 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:53.892207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:53.892213 (+ 6us) server_negotiation.cc:407] Connection header received
0808 20:58:53.892236 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:53.892236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:53.892248 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:53.893092 (+ 844us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:53.893139 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:53.893884 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:53.895425 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:53.895749 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:53.895829 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:53.898288 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:53.898290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:53.898290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:53.898299 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:53.900070 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:53.900529 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:53.900531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:53.900531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:53.900603 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:53.900896 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:53.900897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:53.900897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:53.901026 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:53.901124 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:53.901423 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 20:58:53.901530 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":89,"threads_started":1}
I20260808 20:58:53.901839 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:53.891107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36281 (local address 127.15.232.129:42695)
0808 20:58:53.891271 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:53.891319 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:53.892013 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:53.892416 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:53.892420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:53.892688 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:53.892974 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:53.892981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:53.894009 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:53.894010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:53.895316 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:53.895319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:53.895862 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:53.895863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:53.896002 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:53.896614 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:53.896623 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:53.898178 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:53.900176 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:53.900184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:53.900186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:53.900439 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:53.900683 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:53.900684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:53.900684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:53.900814 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:53.901113 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:53.901120 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:53.901281 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 20:58:53.901400 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 20:58:53.901561 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260808 20:58:53.902503 2050 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36281
I20260808 20:58:53.902639 2050 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:53.902853 2050 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
I20260808 20:58:53.903509 1742 ts_manager.cc:194] Registered new tserver with Master: 3c0561648e984cb59cd77e562785ca8f (127.15.232.129:32883)
I20260808 20:58:53.904179 1742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:42695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:58:53.932878 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:35927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36281
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 20:58:54.057783 2058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:54.058145 2058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:54.058207 2058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:54.060053 2058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:54.060177 2058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:54.060256 2058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:58:54.062629 2058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.232.148:35927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:54.063055 2058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:54.063367 2058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:54.069938 2069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:54.070307 2068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:54.071012 2073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:54.072239 2058 server_base.cc:1034] running on GCE node
I20260808 20:58:54.072557 2058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:54.072856 2058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:54.074095 2058 hybrid_clock.cc:648] HybridClock initialized: now 1786222734074069 us; error 55 us; skew 500 ppm
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:54.077530 2058 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:58:54.078089 2058 webserver.cc:460] Webserver started at http://127.15.232.130:44995/ using document root <none> and password file <none>
I20260808 20:58:54.078344 2058 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:54.078392 2058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:54.078522 2058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:54.098814 2058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "52de5d44ba7c451eac575ce61203a175"
format_stamp: "Formatted at 2026-08-08 20:58:54 on dist-test-slave-c12x"
server_key: "b1949aecdd32b1974a4192e7e04f465f"
server_key_iv: "8b024e73bf3a0bf7fbc910fc37bc8721"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:54.099332 2058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "52de5d44ba7c451eac575ce61203a175"
format_stamp: "Formatted at 2026-08-08 20:58:54 on dist-test-slave-c12x"
server_key: "b1949aecdd32b1974a4192e7e04f465f"
server_key_iv: "8b024e73bf3a0bf7fbc910fc37bc8721"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:54.101459 2058 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 20:58:54.115957 2085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:54.116191 2058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:58:54.116345 2058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "52de5d44ba7c451eac575ce61203a175"
format_stamp: "Formatted at 2026-08-08 20:58:54 on dist-test-slave-c12x"
server_key: "b1949aecdd32b1974a4192e7e04f465f"
server_key_iv: "8b024e73bf3a0bf7fbc910fc37bc8721"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:54.116461 2058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:54.143052 2058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:54.145632 2058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:54.145937 2058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:54.146292 2058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:58:54.146752 2058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:58:54.146847 2058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:54.146929 2058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:58:54.146988 2058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:54.154476 2058 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:39315
I20260808 20:58:54.155557 2058 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:58:54.158218 2248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:39315 every 8 connection(s)
I20260808 20:58:54.161581 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 2058
I20260808 20:58:54.161700 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:58:54.161924 16290 external_mini_cluster.cc:1428] Setting key 9bbeb0c6f7189bbd606bb8cdca656c75
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:54.174553 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:54.159320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:53583 (local address 127.15.232.190:36281)
0808 20:58:54.159375 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:54.159377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:54.159798 (+ 421us) server_negotiation.cc:407] Connection header received
0808 20:58:54.160495 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:54.160497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:54.160514 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:54.160583 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:54.161364 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:54.162191 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:54.167595 (+ 5404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:54.167905 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:54.167960 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:54.170873 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:54.170874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:54.170875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:54.170884 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:54.172757 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:54.173326 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:54.173327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:54.173327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:54.173388 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:54.173697 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:54.173698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:54.173698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:54.173852 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:54.173901 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:54.174321 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 20:58:54.174421 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:58:54.175424 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:54.158989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36281 (local address 127.15.232.130:53583)
0808 20:58:54.159629 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:54.159682 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:54.160369 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:54.160661 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:54.160665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:54.160946 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:54.161248 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:54.161255 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:54.166036 (+ 4781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:54.166037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:54.167458 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:54.167463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:54.168083 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:54.168084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:54.168235 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:54.168915 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:54.168927 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:54.170735 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:54.172867 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:54.172876 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:54.172878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:54.173218 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:54.173457 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:54.173458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:54.173458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:54.173614 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:54.173958 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:54.173964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:54.174146 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 20:58:54.174984 (+ 838us) client_negotiation.cc:240] Negotiation successful
0808 20:58:54.175176 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":107,"threads_started":1}
I20260808 20:58:54.176524 2249 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36281
I20260808 20:58:54.176651 2249 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:54.176896 2249 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
I20260808 20:58:54.177400 1742 ts_manager.cc:194] Registered new tserver with Master: 52de5d44ba7c451eac575ce61203a175 (127.15.232.130:39315)
I20260808 20:58:54.177913 1742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:53583
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:58:54.211336 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:35927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36281
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:58:54.346216 2257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:54.346632 2257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:54.346740 2257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:54.348874 2257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:54.349172 2257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:54.349289 2257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:58:54.351817 2257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.232.148:35927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:54.352420 2257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:54.352797 2257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:54.360388 2264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:54.360869 2268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:54.362186 2257 server_base.cc:1034] running on GCE node
W20260808 20:58:54.365815 2263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:54.366434 2257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:54.366788 2257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:54.367975 2257 hybrid_clock.cc:648] HybridClock initialized: now 1786222734367947 us; error 50 us; skew 500 ppm
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:54.371845 2257 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:58:54.372531 2257 webserver.cc:460] Webserver started at http://127.15.232.131:33935/ using document root <none> and password file <none>
I20260808 20:58:54.372885 2257 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:54.373018 2257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:54.373231 2257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:54.391312 2257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "6651901dfc2d4b02a13d5436518128f3"
format_stamp: "Formatted at 2026-08-08 20:58:54 on dist-test-slave-c12x"
server_key: "55564d6e0667e369bd11c1b38fcde36c"
server_key_iv: "79961aa623dbe473683a859fa212bff6"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:54.391856 2257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "6651901dfc2d4b02a13d5436518128f3"
format_stamp: "Formatted at 2026-08-08 20:58:54 on dist-test-slave-c12x"
server_key: "55564d6e0667e369bd11c1b38fcde36c"
server_key_iv: "79961aa623dbe473683a859fa212bff6"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:54.393464 2257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:58:54.406630 2275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:54.406917 2257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:58:54.407027 2257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "6651901dfc2d4b02a13d5436518128f3"
format_stamp: "Formatted at 2026-08-08 20:58:54 on dist-test-slave-c12x"
server_key: "55564d6e0667e369bd11c1b38fcde36c"
server_key_iv: "79961aa623dbe473683a859fa212bff6"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:54.407166 2257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:54.438334 2257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:54.441076 2257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:54.441308 2257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:54.441668 2257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:58:54.442137 2257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:58:54.442250 2257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:54.442337 2257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:58:54.442404 2257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:54.455917 2257 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:39489
I20260808 20:58:54.456540 2257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:58:54.457139 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 2257
I20260808 20:58:54.457264 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:58:54.457489 16290 external_mini_cluster.cc:1428] Setting key 7f7c67442c4dc943973beb99a5e7c946
I20260808 20:58:54.463275 2445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:39489 every 8 connection(s)
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:54.480326 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:54.464350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:41611 (local address 127.15.232.190:36281)
0808 20:58:54.464432 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:54.464434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:54.464687 (+ 253us) server_negotiation.cc:407] Connection header received
0808 20:58:54.465522 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:54.465523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:54.465556 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:54.465592 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:54.466901 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:54.467735 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:54.469971 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:54.470256 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:54.470339 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:54.475822 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:54.475824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:54.475825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:54.475835 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:54.477740 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:54.478355 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:54.478357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:54.478357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:54.478431 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:54.478785 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:54.478786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:54.478786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:54.478936 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:54.478982 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:54.480124 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0808 20:58:54.480224 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 20:58:54.481153 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:54.464334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36281 (local address 127.15.232.131:41611)
0808 20:58:54.464552 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:54.464599 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:54.465390 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:54.465826 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:54.465831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:54.466281 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:54.466771 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:54.466779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:54.467860 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:54.467861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:54.469843 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:54.469847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:54.471544 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:54.471546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:54.471706 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:54.472830 (+ 1124us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:54.472840 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:54.475692 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:54.477848 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:54.477859 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:54.477862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:54.478253 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:54.478525 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:54.478526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:54.478527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:54.478697 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:54.479033 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:54.479039 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:54.479932 (+ 893us) client_negotiation.cc:769] Sending connection context
0808 20:58:54.480700 (+ 768us) client_negotiation.cc:240] Negotiation successful
0808 20:58:54.480893 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":139,"threads_started":1}
I20260808 20:58:54.481868 2447 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36281
I20260808 20:58:54.481972 2447 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:54.482174 2447 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
I20260808 20:58:54.482604 1742 ts_manager.cc:194] Registered new tserver with Master: 6651901dfc2d4b02a13d5436518128f3 (127.15.232.131:39489)
I20260808 20:58:54.483108 1742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:41611
I20260808 20:58:54.490738 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:58:54.491690 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 1861
I20260808 20:58:54.527846 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 2058
I20260808 20:58:54.538539 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 2257
I20260808 20:58:54.560400 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 1698
I20260808 20:58:54.575277 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36281
--webserver_interface=127.15.232.190
--webserver_port=35777
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:36281
--ranger_config_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:35927/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.232.148:35927/kms with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:58:54.661451 2450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:54.661672 2450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:54.661696 2450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:54.662892 2450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:58:54.662940 2450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:54.662953 2450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:54.662966 2450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:58:54.662979 2450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:58:54.664739 2450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.232.148:35927/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:36281
--ranger_config_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=35777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:54.665016 2450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:54.665261 2450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:54.670441 2459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:54.670480 2456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:54.670666 2450 server_base.cc:1034] running on GCE node
W20260808 20:58:54.670446 2464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:54.671147 2450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:54.671504 2450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:54.672662 2450 hybrid_clock.cc:648] HybridClock initialized: now 1786222734672651 us; error 29 us; skew 500 ppm
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:54.675279 2450 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:58:54.675861 2450 webserver.cc:460] Webserver started at http://127.15.232.190:35777/ using document root <none> and password file <none>
I20260808 20:58:54.676136 2450 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:54.676195 2450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:54.750492 2450 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:58:54.751291 2473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:54.751502 2450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:58:54.751596 2450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "5338321e91fd4602acdc00d67a9a732d"
format_stamp: "Formatted at 2026-08-08 20:58:52 on dist-test-slave-c12x"
server_key: "0bcb45746d23a7097db26d58186c06f4"
server_key_iv: "cad3f84292c28e6df27e23201d89fc9e"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:54.752005 2450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:54.771136 2450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:54.773375 2450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:54.773572 2450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:54.778735 2450 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:36281
I20260808 20:58:54.778765 2546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:36281 every 8 connection(s)
I20260808 20:58:54.779228 2450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:58:54.780568 2547 sys_catalog.cc:263] Verifying existing consensus state
I20260808 20:58:54.781263 2547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Bootstrap starting.
I20260808 20:58:54.783831 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 2450
I20260808 20:58:54.784152 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:35927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:32883
--local_ip_for_outbound_sockets=127.15.232.129
--tserver_master_addrs=127.15.232.190:36281
--webserver_port=45187
--webserver_interface=127.15.232.129
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.232.148:35927/kms with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:58:54.790349 2547 log.cc:824] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:54.792277 2547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:58:54.792600 2547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Bootstrap complete.
I20260808 20:58:54.795082 2547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } }
I20260808 20:58:54.795349 2547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5338321e91fd4602acdc00d67a9a732d, State: Initialized, Role: FOLLOWER
I20260808 20:58:54.795571 2547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } }
I20260808 20:58:54.795687 2547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:58:54.795768 2547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:58:54.795850 2547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 1 FOLLOWER]: Advancing to term 2
I20260808 20:58:54.796900 2547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } }
I20260808 20:58:54.797037 2547 leader_election.cc:302] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5338321e91fd4602acdc00d67a9a732d; no voters:
I20260808 20:58:54.797281 2547 leader_election.cc:288] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 20:58:54.797319 2551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 2 FOLLOWER]: Leader election won for term 2
I20260808 20:58:54.797597 2551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [term 2 LEADER]: Becoming Leader. State: Replica: 5338321e91fd4602acdc00d67a9a732d, State: Running, Role: LEADER
I20260808 20:58:54.797650 2547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:58:54.798187 2551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } }
I20260808 20:58:54.798514 2552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5338321e91fd4602acdc00d67a9a732d. Latest consensus state: current_term: 2 leader_uuid: "5338321e91fd4602acdc00d67a9a732d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } } }
I20260808 20:58:54.798683 2552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: This master's current role is: LEADER
I20260808 20:58:54.798848 2554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5338321e91fd4602acdc00d67a9a732d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5338321e91fd4602acdc00d67a9a732d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 36281 } } }
I20260808 20:58:54.799011 2554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d [sys.catalog]: This master's current role is: LEADER
I20260808 20:58:54.799226 2561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:58:54.799759 2561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:58:54.799892 2561 catalog_manager.cc:1259] Loaded cluster ID: 41265925d719411f8a91ddee82d38ff8
I20260808 20:58:54.799978 2561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:58:54.800200 2547 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 20:58:54.801285 2561 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:58:54.801462 2561 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Loaded TSK: 0
I20260808 20:58:54.801681 2561 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 20:58:54.889599 2549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:54.889904 2549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:54.889950 2549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:54.891323 2549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:54.891388 2549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:54.891438 2549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:58:54.893077 2549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.232.148:35927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:32883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=45187
--tserver_master_addrs=127.15.232.190:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:54.893348 2549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:54.893657 2549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:54.899394 2601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:54.899370 2603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:54.899484 2599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:54.900087 2549 server_base.cc:1034] running on GCE node
I20260808 20:58:54.900317 2549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:54.900641 2549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:54.901834 2549 hybrid_clock.cc:648] HybridClock initialized: now 1786222734901797 us; error 50 us; skew 500 ppm
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:54.905292 2549 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:58:54.905965 2549 webserver.cc:460] Webserver started at http://127.15.232.129:45187/ using document root <none> and password file <none>
I20260808 20:58:54.906297 2549 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:54.906435 2549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 20:58:54 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:54.944792 2549 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 20:58:54.945977 2612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:54.946201 2549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:58:54.946306 2549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "3c0561648e984cb59cd77e562785ca8f"
format_stamp: "Formatted at 2026-08-08 20:58:53 on dist-test-slave-c12x"
server_key: "1aba685b72d8b35d6ba1416ddc80f9d9"
server_key_iv: "756b9401ea46bb7fd8e0030161684b81"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:54.946810 2549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:54.989244 2549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:54.991319 2549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:54.991525 2549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:54.991825 2549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:58:54.992228 2549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:58:54.992275 2549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:54.992314 2549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:58:54.992338 2549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:55.000186 2549 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:32883
I20260808 20:58:55.000638 2784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:32883 every 8 connection(s)
I20260808 20:58:55.000735 2549 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:58:55.004320 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 2549
I20260808 20:58:55.004657 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:35927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:39315
--local_ip_for_outbound_sockets=127.15.232.130
--tserver_master_addrs=127.15.232.190:36281
--webserver_port=44995
--webserver_interface=127.15.232.130
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.232.148:35927/kms with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 20:58:55 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222734, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:55.017530 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:55.001684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36281 (local address 127.15.232.129:49781)
0808 20:58:55.001879 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:55.001924 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:55.002576 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:55.003249 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:55.003253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:55.003534 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:55.003826 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:55.003833 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.007813 (+ 3980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.007814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:55.009206 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:55.009210 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.010143 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.010145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:55.010294 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:55.010938 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:55.010949 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:55.012844 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:55.015364 (+ 2520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:55.015372 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:55.015375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:55.015740 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:55.016090 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:55.016091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:55.016091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:55.016241 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:55.016815 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:55.016822 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:55.016987 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 20:58:55.017098 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 20:58:55.017277 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
I20260808 20:58:55.018644 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:55.001810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:49781 (local address 127.15.232.190:36281)
0808 20:58:55.002038 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:55.002046 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:55.002089 (+ 43us) server_negotiation.cc:407] Connection header received
0808 20:58:55.002842 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:55.002848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:55.002931 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:55.003067 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:55.003968 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.007650 (+ 3682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.009343 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.009985 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.010618 (+ 633us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:55.012989 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:55.012994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:55.012995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:55.013008 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:55.015248 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:55.015854 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:55.015857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:55.015858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:55.015984 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:55.016328 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:55.016329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:55.016330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:55.016706 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:55.018204 (+ 1498us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:55.018284 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 20:58:55.018428 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
I20260808 20:58:55.019690 2785 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36281
I20260808 20:58:55.019793 2785 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:55.019999 2785 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
W20260808 20:58:55.021024 2785 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:36281 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 20:58:55.021122 2785 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:55.021191 2785 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
I20260808 20:58:55.021625 2785 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:55.021701 2785 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
W20260808 20:58:55.022053 2785 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 20:58:55.119540 2790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:55.119756 2790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:55.119782 2790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:55.121400 2790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:55.121472 2790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:55.121516 2790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:58:55.124382 2790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.232.148:35927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:39315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=44995
--tserver_master_addrs=127.15.232.190:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:55.124677 2790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:55.124965 2790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:55.132577 2804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:55.132908 2799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:55.132987 2801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:55.134166 2790 server_base.cc:1034] running on GCE node
I20260808 20:58:55.134404 2790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:55.134711 2790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:55.135900 2790 hybrid_clock.cc:648] HybridClock initialized: now 1786222735135876 us; error 46 us; skew 500 ppm
Aug 08 20:58:55 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222735, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:55.138895 2790 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:58:55.139498 2790 webserver.cc:460] Webserver started at http://127.15.232.130:44995/ using document root <none> and password file <none>
I20260808 20:58:55.139756 2790 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:55.139839 2790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 20:58:55 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222735, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:55.155911 2790 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:58:55.161438 2816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:55.162881 2790 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.001s
I20260808 20:58:55.163034 2790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "52de5d44ba7c451eac575ce61203a175"
format_stamp: "Formatted at 2026-08-08 20:58:54 on dist-test-slave-c12x"
server_key: "b1949aecdd32b1974a4192e7e04f465f"
server_key_iv: "8b024e73bf3a0bf7fbc910fc37bc8721"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:55.163637 2790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:55.186194 2790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:55.189129 2790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:55.189322 2790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:55.189615 2790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:58:55.190016 2790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:58:55.190084 2790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:55.190117 2790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:58:55.190141 2790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:55.197847 2790 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:39315
I20260808 20:58:55.197949 2980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:39315 every 8 connection(s)
I20260808 20:58:55.198385 2790 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:58:55.200227 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 2790
I20260808 20:58:55.200538 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.232.148:35927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:39489
--local_ip_for_outbound_sockets=127.15.232.131
--tserver_master_addrs=127.15.232.190:36281
--webserver_port=33935
--webserver_interface=127.15.232.131
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.232.148:35927/kms with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 20:58:55 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222735, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:55.212754 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:55.199951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:46233 (local address 127.15.232.190:36281)
0808 20:58:55.200001 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:55.200004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:55.200296 (+ 292us) server_negotiation.cc:407] Connection header received
0808 20:58:55.201558 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:55.201559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:55.201577 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:55.201610 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:55.202511 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.203250 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.204590 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.204910 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.205000 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:55.208517 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:55.208520 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:55.208520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:55.208532 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:55.210680 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:55.211344 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:55.211345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:55.211346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:55.211419 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:55.211784 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:55.211785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:55.211785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:55.211958 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:55.212038 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:55.212491 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 20:58:55.212606 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 20:58:55.213493 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:55.199934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36281 (local address 127.15.232.130:46233)
0808 20:58:55.200139 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:55.200193 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:55.201409 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:55.201807 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:55.201811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:55.202079 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:55.202373 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:55.202379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.203379 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.203380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:55.204482 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:55.204486 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.205058 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.205059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:55.205240 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:55.205937 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:55.205949 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:55.208362 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:55.210804 (+ 2442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:55.210817 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:55.210821 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:55.211235 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:55.211519 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:55.211520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:55.211521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:55.211693 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:55.212084 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:55.212093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:55.212303 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 20:58:55.212968 (+ 665us) client_negotiation.cc:240] Negotiation successful
0808 20:58:55.213178 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":108,"threads_started":1}
I20260808 20:58:55.214001 2981 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36281
I20260808 20:58:55.214136 2981 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:55.214356 2981 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
W20260808 20:58:55.215070 2981 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:36281 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 20:58:55.215162 2981 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:55.215240 2981 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
I20260808 20:58:55.215684 2981 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:55.215772 2981 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
W20260808 20:58:55.216141 2981 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 20:58:55.324869 2984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:55.325171 2984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:55.325212 2984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:55.327010 2984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:55.327086 2984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:55.327131 2984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:58:55.329480 2984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.232.148:35927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:39489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=33935
--tserver_master_addrs=127.15.232.190:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:55.329790 2984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:55.330070 2984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:55.335945 2993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:55.336275 2994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:55.336436 2984 server_base.cc:1034] running on GCE node
W20260808 20:58:55.339658 3000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:55.339978 2984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:55.340350 2984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:55.341627 2984 hybrid_clock.cc:648] HybridClock initialized: now 1786222735341609 us; error 40 us; skew 500 ppm
Aug 08 20:58:55 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222735, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:55.345203 2984 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:58:55.345800 2984 webserver.cc:460] Webserver started at http://127.15.232.131:33935/ using document root <none> and password file <none>
I20260808 20:58:55.346101 2984 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:55.346184 2984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 20:58:55 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222735, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:55.367911 2984 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 20:58:55.369012 3012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:55.369194 2984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:58:55.369278 2984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "6651901dfc2d4b02a13d5436518128f3"
format_stamp: "Formatted at 2026-08-08 20:58:54 on dist-test-slave-c12x"
server_key: "55564d6e0667e369bd11c1b38fcde36c"
server_key_iv: "79961aa623dbe473683a859fa212bff6"
server_key_version: "kuduclusterkey@0"
I20260808 20:58:55.369778 2984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:55.408051 2984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:55.411036 2984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:55.411227 2984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:55.411548 2984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:58:55.411907 2984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:58:55.411962 2984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:55.412007 2984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:58:55.412034 2984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:55.424803 2984 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:39489
I20260808 20:58:55.424863 3179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:39489 every 8 connection(s)
I20260808 20:58:55.425360 2984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:58:55.433702 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 2984
Aug 08 20:58:55 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222735, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:55.441857 3182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:55.427429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:36281 (local address 127.15.232.131:40373)
0808 20:58:55.427906 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:55.427958 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:55.428641 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:55.429048 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:55.429053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:55.429338 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:55.429658 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:55.429665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.430735 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.430736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:55.432108 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:55.432111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.432706 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.432707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:55.432852 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:55.434042 (+ 1190us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:55.434053 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:55.437881 (+ 3828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:55.440016 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:55.440025 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:55.440028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:55.440358 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:55.440639 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:55.440640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:55.440640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:55.440799 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:55.441139 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:55.441145 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:55.441325 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 20:58:55.441435 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 20:58:55.441610 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":117,"threads_started":1}
I20260808 20:58:55.442520 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:55.427592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:40373 (local address 127.15.232.190:36281)
0808 20:58:55.427649 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:55.427651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:55.428078 (+ 427us) server_negotiation.cc:407] Connection header received
0808 20:58:55.428789 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:55.428790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:55.428805 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:55.428838 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:55.429782 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.430539 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.432225 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.432524 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.432585 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:55.438097 (+ 5512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:55.438099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:55.438099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:55.438107 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:55.439901 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:55.440473 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:55.440475 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:55.440475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:55.440549 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:55.440884 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:55.440886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:55.440886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:55.441041 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:55.441108 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:55.442342 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0808 20:58:55.442437 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 20:58:55.442951 3180 heartbeater.cc:347] Connected to a master server at 127.15.232.190:36281
I20260808 20:58:55.443070 3180 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:55.443293 3180 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
W20260808 20:58:55.444038 3180 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:36281 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 20:58:55.444144 3180 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:55.444221 3180 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
I20260808 20:58:55.444628 3180 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:55.444705 3180 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
W20260808 20:58:55.445066 3180 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 20:58:55.446241 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:55.434302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35108 (local address 127.15.232.190:36281)
0808 20:58:55.438534 (+ 4232us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:55.438536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:55.438543 (+ 7us) server_negotiation.cc:407] Connection header received
0808 20:58:55.438566 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:55.438567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:55.438579 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:55.438649 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:55.439680 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.440389 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.441163 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:55.441370 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:55.441459 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:55.442895 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:55.442896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:55.442897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:55.442904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:55.445053 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:55.445434 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:55.445436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:55.445436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:55.445503 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:55.445712 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:55.445713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:55.445713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:55.445830 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:55.446006 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:55.446048 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 20:58:55.446133 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4200,"thread_start_us":102,"threads_started":1}
Aug 08 20:58:55 dist-test-slave-c12x krb5kdc[1067](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786222735, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:56.022292 2785 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:56.022492 2785 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
I20260808 20:58:56.022986 2496 ts_manager.cc:194] Registered new tserver with Master: 3c0561648e984cb59cd77e562785ca8f (127.15.232.129:32883)
I20260808 20:58:56.024075 2496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:49781
I20260808 20:58:56.216362 2981 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:56.216539 2981 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
I20260808 20:58:56.216953 2496 ts_manager.cc:194] Registered new tserver with Master: 52de5d44ba7c451eac575ce61203a175 (127.15.232.130:39315)
I20260808 20:58:56.217489 2496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:46233
Aug 08 20:58:56 dist-test-slave-c12x krb5kdc[1067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222735, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for HTTP/127.15.232.148@KRBTEST.COM
I20260808 20:58:56.445269 3180 heartbeater.cc:464] Registering TS with master...
I20260808 20:58:56.445444 3180 heartbeater.cc:511] Master 127.15.232.190:36281 requested a full tablet report, sending...
I20260808 20:58:56.445847 2496 ts_manager.cc:194] Registered new tserver with Master: 6651901dfc2d4b02a13d5436518128f3 (127.15.232.131:39489)
I20260808 20:58:56.446421 2496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:40373
I20260808 20:58:56.453608 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:58:56.465678 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.457516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.15.232.190:36281)
0808 20:58:56.457638 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:56.457640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:56.457651 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:58:56.457680 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:56.457681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:56.457694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:56.457776 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:56.458530 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.459320 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.460419 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.460731 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.460815 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.462327 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:56.462329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:56.462329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:56.462337 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:56.464346 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:56.464781 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:56.464782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:56.464782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:56.464856 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:56.465087 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:56.465088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:56.465088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:56.465237 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:56.465364 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:56.465470 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 20:58:56.465566 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260808 20:58:56.467973 2496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:58:56.470395 2496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:58:56.480461 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.474257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:32883 (local address 127.0.0.1:50478)
0808 20:58:56.475097 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:56.475106 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:56.475160 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:56.475527 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:56.475530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:56.475533 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:56.475614 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.475616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.477213 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.477214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.478598 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.478601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.480130 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.480131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.480221 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.480236 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 20:58:56.480306 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 20:58:56.480360 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"thread_start_us":74,"threads_started":1}
I20260808 20:58:56.480921 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.474727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50478 (local address 127.15.232.129:32883)
0808 20:58:56.474912 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:56.474917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:56.475220 (+ 303us) server_negotiation.cc:407] Connection header received
0808 20:58:56.475261 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:56.475263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:56.475279 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:56.475386 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:56.475699 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.477129 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.478695 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.479951 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.480032 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.480091 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:56.480800 (+ 709us) server_negotiation.cc:299] Negotiation successful
0808 20:58:56.480844 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":75,"threads_started":1}
I20260808 20:58:56.481930 2661 tablet_service.cc:1467] Processing CreateTablet for tablet b098f2232792461e8eab2b710d5f79c1 (DEFAULT_TABLE table=test-table [id=b3a59c9e00d6422a81a091f1c788b056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:58:56.482337 2661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b098f2232792461e8eab2b710d5f79c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:56.482790 3218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.473998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:39315 (local address 127.0.0.1:45042)
0808 20:58:56.474450 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:56.474492 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:56.474559 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:56.475659 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:56.475660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:56.475663 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:56.475710 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.475712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.480400 (+ 4688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.480401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.481517 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.481521 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.482545 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.482546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.482612 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.482616 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:58:56.482666 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 20:58:56.482708 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":104,"threads_started":1}
I20260808 20:58:56.483198 3219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.474014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:39489 (local address 127.0.0.1:57780)
0808 20:58:56.474992 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:56.475004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:56.475067 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:56.475785 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:56.475785 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:56.475787 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:56.475813 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.475815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.478148 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.478149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.479264 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.479267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.482952 (+ 3685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.482953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.483029 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.483034 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:58:56.483095 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 20:58:56.483132 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"thread_start_us":104,"threads_started":1}
I20260808 20:58:56.483500 3220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.474294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57780 (local address 127.15.232.131:39489)
0808 20:58:56.475031 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:56.475035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:56.475045 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:58:56.475169 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:56.475172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:56.475189 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:56.475263 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:56.475892 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.476892 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.479372 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.480187 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.480271 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.480332 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:56.483348 (+ 3016us) server_negotiation.cc:299] Negotiation successful
0808 20:58:56.483404 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":57,"threads_started":1}
I20260808 20:58:56.483959 3221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.474545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.15.232.130:39315)
0808 20:58:56.475399 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:56.475404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:56.475413 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:58:56.475480 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:56.475482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:56.475496 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:56.475572 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:56.476996 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.478045 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.481628 (+ 3583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.482371 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.482445 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.482505 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:56.483821 (+ 1316us) server_negotiation.cc:299] Negotiation successful
0808 20:58:56.483871 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":69,"threads_started":1}
I20260808 20:58:56.484239 3080 tablet_service.cc:1467] Processing CreateTablet for tablet b098f2232792461e8eab2b710d5f79c1 (DEFAULT_TABLE table=test-table [id=b3a59c9e00d6422a81a091f1c788b056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:58:56.484633 3080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b098f2232792461e8eab2b710d5f79c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:56.484606 2852 tablet_service.cc:1467] Processing CreateTablet for tablet b098f2232792461e8eab2b710d5f79c1 (DEFAULT_TABLE table=test-table [id=b3a59c9e00d6422a81a091f1c788b056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:58:56.484941 2852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b098f2232792461e8eab2b710d5f79c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:56.488531 3226 tablet_bootstrap.cc:492] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: Bootstrap starting.
I20260808 20:58:56.489054 3228 tablet_bootstrap.cc:492] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: Bootstrap starting.
I20260808 20:58:56.489362 3226 tablet_bootstrap.cc:654] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:56.489799 3228 tablet_bootstrap.cc:654] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:56.489882 3226 log.cc:824] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:56.490197 3228 log.cc:824] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:56.490804 3226 tablet_bootstrap.cc:492] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: No bootstrap required, opened a new log
I20260808 20:58:56.490875 3226 ts_tablet_manager.cc:1397] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:58:56.491024 3228 tablet_bootstrap.cc:492] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: No bootstrap required, opened a new log
I20260808 20:58:56.491094 3228 ts_tablet_manager.cc:1397] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 20:58:56.493527 3226 raft_consensus.cc:348] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } }
I20260808 20:58:56.493713 3226 raft_consensus.cc:374] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:56.493744 3226 raft_consensus.cc:729] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6651901dfc2d4b02a13d5436518128f3, State: Initialized, Role: FOLLOWER
I20260808 20:58:56.494055 3228 raft_consensus.cc:348] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } }
I20260808 20:58:56.494329 3228 raft_consensus.cc:374] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:56.494422 3228 raft_consensus.cc:729] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52de5d44ba7c451eac575ce61203a175, State: Initialized, Role: FOLLOWER
I20260808 20:58:56.494518 3226 consensus_queue.cc:260] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } }
I20260808 20:58:56.494614 3228 consensus_queue.cc:260] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } }
I20260808 20:58:56.494982 3228 ts_tablet_manager.cc:1428] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 20:58:56.494982 3226 ts_tablet_manager.cc:1428] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 20:58:56.495128 3180 heartbeater.cc:503] Master 127.15.232.190:36281 was elected leader, sending a full tablet report...
I20260808 20:58:56.495163 2981 heartbeater.cc:503] Master 127.15.232.190:36281 was elected leader, sending a full tablet report...
I20260808 20:58:56.499643 3224 tablet_bootstrap.cc:492] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f: Bootstrap starting.
I20260808 20:58:56.500458 3224 tablet_bootstrap.cc:654] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:56.500801 3224 log.cc:824] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:56.501699 3224 tablet_bootstrap.cc:492] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f: No bootstrap required, opened a new log
I20260808 20:58:56.501792 3224 ts_tablet_manager.cc:1397] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:58:56.504299 3224 raft_consensus.cc:348] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } }
I20260808 20:58:56.504544 3224 raft_consensus.cc:374] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:56.504623 3224 raft_consensus.cc:729] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c0561648e984cb59cd77e562785ca8f, State: Initialized, Role: FOLLOWER
I20260808 20:58:56.504828 3224 consensus_queue.cc:260] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } }
I20260808 20:58:56.505208 3224 ts_tablet_manager.cc:1428] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 20:58:56.505266 2785 heartbeater.cc:503] Master 127.15.232.190:36281 was elected leader, sending a full tablet report...
I20260808 20:58:56.534989 3238 raft_consensus.cc:482] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:58:56.535172 3238 raft_consensus.cc:504] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } }
I20260808 20:58:56.535634 3238 leader_election.cc:288] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6651901dfc2d4b02a13d5436518128f3 (127.15.232.131:39489), 3c0561648e984cb59cd77e562785ca8f (127.15.232.129:32883)
I20260808 20:58:56.539997 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.536122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:49057 (local address 127.15.232.129:32883)
0808 20:58:56.536168 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:56.536170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:56.536329 (+ 159us) server_negotiation.cc:407] Connection header received
0808 20:58:56.536435 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:56.536436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:56.536446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:56.536475 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:56.536696 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.537411 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.538989 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.539566 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.539616 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.539625 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:56.539877 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 20:58:56.539924 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:58:56.540035 3220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.536134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:43703 (local address 127.15.232.131:39489)
0808 20:58:56.536181 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:56.536183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:56.536332 (+ 149us) server_negotiation.cc:407] Connection header received
0808 20:58:56.536442 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:56.536443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:56.536453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:56.536484 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:58:56.536727 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.537456 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.538991 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.539622 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.539676 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.539684 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:56.539917 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 20:58:56.539959 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:58:56.540319 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.535821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:39489 (local address 127.15.232.130:43703)
0808 20:58:56.536256 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:56.536268 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:56.536364 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:56.536558 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:56.536559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:56.536561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:56.536642 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.536644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.537557 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.537558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.538879 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.538883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.539746 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.539748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.539836 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.539839 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:58:56.540170 (+ 331us) client_negotiation.cc:240] Negotiation successful
0808 20:58:56.540221 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":3,"thread_start_us":124,"threads_started":1}
I20260808 20:58:56.540669 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.535844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:32883 (local address 127.15.232.130:49057)
0808 20:58:56.536258 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:56.536268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:56.536359 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:56.536542 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:56.536543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:56.536545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:58:56.536625 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.536628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.537508 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.537509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.538878 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:56.538882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.539720 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.539721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:56.539794 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.539799 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:58:56.540535 (+ 736us) client_negotiation.cc:240] Negotiation successful
0808 20:58:56.540583 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":82,"thread_start_us":106,"threads_started":1}
I20260808 20:58:56.540726 3113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b098f2232792461e8eab2b710d5f79c1" candidate_uuid: "52de5d44ba7c451eac575ce61203a175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6651901dfc2d4b02a13d5436518128f3" is_pre_election: true
I20260808 20:58:56.540901 3113 raft_consensus.cc:2454] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52de5d44ba7c451eac575ce61203a175 in term 0.
I20260808 20:58:56.541141 2710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b098f2232792461e8eab2b710d5f79c1" candidate_uuid: "52de5d44ba7c451eac575ce61203a175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c0561648e984cb59cd77e562785ca8f" is_pre_election: true
I20260808 20:58:56.541277 2710 raft_consensus.cc:2454] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52de5d44ba7c451eac575ce61203a175 in term 0.
I20260808 20:58:56.541338 2817 leader_election.cc:302] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52de5d44ba7c451eac575ce61203a175, 6651901dfc2d4b02a13d5436518128f3; no voters:
I20260808 20:58:56.541518 3238 raft_consensus.cc:2781] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:58:56.541561 3238 raft_consensus.cc:482] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:58:56.541584 3238 raft_consensus.cc:3037] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:56.542685 3238 raft_consensus.cc:504] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } }
I20260808 20:58:56.542834 3238 leader_election.cc:288] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [CANDIDATE]: Term 1 election: Requested vote from peers 6651901dfc2d4b02a13d5436518128f3 (127.15.232.131:39489), 3c0561648e984cb59cd77e562785ca8f (127.15.232.129:32883)
I20260808 20:58:56.543045 2710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b098f2232792461e8eab2b710d5f79c1" candidate_uuid: "52de5d44ba7c451eac575ce61203a175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c0561648e984cb59cd77e562785ca8f"
I20260808 20:58:56.543100 3113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b098f2232792461e8eab2b710d5f79c1" candidate_uuid: "52de5d44ba7c451eac575ce61203a175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6651901dfc2d4b02a13d5436518128f3"
I20260808 20:58:56.543141 2710 raft_consensus.cc:3037] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:56.543179 3113 raft_consensus.cc:3037] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:56.544092 3113 raft_consensus.cc:2454] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52de5d44ba7c451eac575ce61203a175 in term 1.
I20260808 20:58:56.544101 2710 raft_consensus.cc:2454] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52de5d44ba7c451eac575ce61203a175 in term 1.
I20260808 20:58:56.544289 2819 leader_election.cc:302] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c0561648e984cb59cd77e562785ca8f, 52de5d44ba7c451eac575ce61203a175; no voters:
I20260808 20:58:56.544404 3238 raft_consensus.cc:2781] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:58:56.544620 3238 raft_consensus.cc:686] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 1 LEADER]: Becoming Leader. State: Replica: 52de5d44ba7c451eac575ce61203a175, State: Running, Role: LEADER
I20260808 20:58:56.544725 3238 consensus_queue.cc:237] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } }
I20260808 20:58:56.545574 2494 catalog_manager.cc:5495] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52de5d44ba7c451eac575ce61203a175 (127.15.232.130). New cstate: current_term: 1 leader_uuid: "52de5d44ba7c451eac575ce61203a175" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52de5d44ba7c451eac575ce61203a175" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 39315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 } health_report { overall_health: UNKNOWN } } }
I20260808 20:58:56.562492 3221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.559096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.15.232.130:39315)
0808 20:58:56.559139 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:56.559141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:56.559151 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:58:56.559223 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:56.559224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:56.559245 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:56.559324 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:58:56.559510 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.560237 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.561371 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:56.561664 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:56.561734 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:56.561935 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:58:56.562087 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:58:56.562197 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:56.562373 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 20:58:56.562402 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:58:56.565129 2710 raft_consensus.cc:1261] T b098f2232792461e8eab2b710d5f79c1 P 3c0561648e984cb59cd77e562785ca8f [term 1 FOLLOWER]: Refusing update from remote peer 52de5d44ba7c451eac575ce61203a175: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:58:56.565197 3113 raft_consensus.cc:1261] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [term 1 FOLLOWER]: Refusing update from remote peer 52de5d44ba7c451eac575ce61203a175: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:58:56.565547 3238 consensus_queue.cc:1035] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6651901dfc2d4b02a13d5436518128f3" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:58:56.565733 3244 consensus_queue.cc:1035] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c0561648e984cb59cd77e562785ca8f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 32883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:58:56.568992 3248 mvcc.cc:205] Tried to move back new op lower bound from 7316368328967520256 to 7316368328887631872. Current Snapshot: MvccSnapshot[applied={T|T < 7316368328967520256}]
I20260808 20:58:56.574179 2494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:58:56.574291 2494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:58:56.574697 3250 mvcc.cc:205] Tried to move back new op lower bound from 7316368328967520256 to 7316368328887631872. Current Snapshot: MvccSnapshot[applied={T|T < 7316368328967520256}]
I20260808 20:58:56.574849 3249 mvcc.cc:205] Tried to move back new op lower bound from 7316368328967520256 to 7316368328887631872. Current Snapshot: MvccSnapshot[applied={T|T < 7316368328967520256}]
I20260808 20:58:56.575317 2494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5338321e91fd4602acdc00d67a9a732d: Sending DeleteTablet for 3 replicas of tablet b098f2232792461e8eab2b710d5f79c1
I20260808 20:58:56.575870 2852 tablet_service.cc:1514] Processing DeleteTablet for tablet b098f2232792461e8eab2b710d5f79c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:58:56 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:45042
I20260808 20:58:56.576409 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 2549
I20260808 20:58:56.583104 3080 tablet_service.cc:1514] Processing DeleteTablet for tablet b098f2232792461e8eab2b710d5f79c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:58:56 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:57780
I20260808 20:58:56.585062 3260 tablet_replica.cc:330] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: stopping tablet replica
I20260808 20:58:56.585209 3260 raft_consensus.cc:2229] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:58:56.585393 3260 raft_consensus.cc:2258] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:58:56.588364 3260 ts_tablet_manager.cc:1905] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:58:56.590231 3260 ts_tablet_manager.cc:1918] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:58:56.590322 3260 log.cc:1196] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/b098f2232792461e8eab2b710d5f79c1
I20260808 20:58:56.590601 3260 ts_tablet_manager.cc:1939] T b098f2232792461e8eab2b710d5f79c1 P 52de5d44ba7c451eac575ce61203a175: Deleting consensus metadata
I20260808 20:58:56.591084 2477 catalog_manager.cc:4841] TS 52de5d44ba7c451eac575ce61203a175 (127.15.232.130:39315): tablet b098f2232792461e8eab2b710d5f79c1 (table test-table [id=b3a59c9e00d6422a81a091f1c788b056]) successfully deleted
I20260808 20:58:56.593621 3261 tablet_replica.cc:330] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: stopping tablet replica
I20260808 20:58:56.593835 3261 raft_consensus.cc:2229] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:58:56.594024 3261 raft_consensus.cc:2258] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:58:56.594611 3261 ts_tablet_manager.cc:1905] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 20:58:56.596589 2477 connection.cc:535] client connection to 127.15.232.129:32883 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 20:58:56.596830 2477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 20:58:56.597556 3261 ts_tablet_manager.cc:1918] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:58:56.597824 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 2790
I20260808 20:58:56.627502 3261 log.cc:1196] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/b098f2232792461e8eab2b710d5f79c1
I20260808 20:58:56.628206 3261 ts_tablet_manager.cc:1939] T b098f2232792461e8eab2b710d5f79c1 P 6651901dfc2d4b02a13d5436518128f3: Deleting consensus metadata
I20260808 20:58:56.628998 2474 catalog_manager.cc:4841] TS 6651901dfc2d4b02a13d5436518128f3 (127.15.232.131:39489): tablet b098f2232792461e8eab2b710d5f79c1 (table test-table [id=b3a59c9e00d6422a81a091f1c788b056]) successfully deleted
I20260808 20:58:56.639788 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 2984
I20260808 20:58:56.640148 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:56.640003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:32883 (local address 127.0.0.1:50492)
0808 20:58:56.640062 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:56.640099 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:32883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 20:58:56.640277 2477 catalog_manager.cc:4584] TS 3c0561648e984cb59cd77e562785ca8f (127.15.232.129:32883): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b098f2232792461e8eab2b710d5f79c1: Network error: Client connection negotiation failed: client connection to 127.15.232.129:32883: connect: Connection refused (error 111)
I20260808 20:58:56.657652 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 2450
I20260808 20:58:56.670965 16290 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 20:58:57.027251 16290 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 20:58:57.027370 16290 mini_ranger.cc:66] Stopping Ranger...
I20260808 20:58:57.395968 16290 mini_ranger.cc:68] Stopped Ranger
2026-08-08 20:58:57.396 UTC [1111] LOG: received smart shutdown request
2026-08-08 20:58:57.401 UTC [1111] LOG: background worker "logical replication launcher" (PID 1123) exited with exit code 1
2026-08-08 20:58:57.401 UTC [1116] LOG: shutting down
2026-08-08 20:58:57.605 UTC [1111] LOG: database system is shut down
2026-08-08T20:58:57Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51710 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:58:57 dist-test-slave-c12x krb5kdc[3274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:58:57 dist-test-slave-c12x krb5kdc[3274](info): set up 2 sockets
Aug 08 20:58:57 dist-test-slave-c12x krb5kdc[3274](info): commencing operation
krb5kdc: starting...
W20260808 20:58:58.982250 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:58:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:58:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:58:59.118863 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:32857
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:32857,127.15.232.189:45709,127.15.232.188:40943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:58:59.206602 3291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:59.206874 3291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:59.206916 3291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:59.208150 3291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:58:59.208233 3291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:59.208259 3291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:59.208289 3291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:58:59.208321 3291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:58:59.209899 3291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:32857,127.15.232.189:45709,127.15.232.188:40943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:32857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:59.210206 3291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:59.210457 3291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:59.216615 3299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:59.216615 3298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:59.216650 3291 server_base.cc:1034] running on GCE node
W20260808 20:58:59.216650 3301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:59.217237 3291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:59.217564 3291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:59.218720 3291 hybrid_clock.cc:648] HybridClock initialized: now 1786222739218705 us; error 35 us; skew 500 ppm
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:59.225005 3291 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:58:59.225508 3291 webserver.cc:460] Webserver started at http://127.15.232.190:42629/ using document root <none> and password file <none>
I20260808 20:58:59.225749 3291 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:59.225798 3291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:59.225950 3291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:58:59.227108 3291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "ea38efa1470a4ce684765299b763f451"
format_stamp: "Formatted at 2026-08-08 20:58:59 on dist-test-slave-c12x"
server_key: "c7e079eefcdb851239da16a97837ed4a"
server_key_iv: "9c5a9e2f6602c56a0a5ba52e97e215ab"
server_key_version: "encryptionkey@0"
I20260808 20:58:59.227535 3291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "ea38efa1470a4ce684765299b763f451"
format_stamp: "Formatted at 2026-08-08 20:58:59 on dist-test-slave-c12x"
server_key: "c7e079eefcdb851239da16a97837ed4a"
server_key_iv: "9c5a9e2f6602c56a0a5ba52e97e215ab"
server_key_version: "encryptionkey@0"
I20260808 20:58:59.228988 3291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:58:59.229833 3309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:59.230058 3291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:58:59.230171 3291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "ea38efa1470a4ce684765299b763f451"
format_stamp: "Formatted at 2026-08-08 20:58:59 on dist-test-slave-c12x"
server_key: "c7e079eefcdb851239da16a97837ed4a"
server_key_iv: "9c5a9e2f6602c56a0a5ba52e97e215ab"
server_key_version: "encryptionkey@0"
I20260808 20:58:59.230249 3291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:59.242555 3291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:59.245316 3291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:59.245453 3291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:59.251798 3291 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:32857
I20260808 20:58:59.252089 3380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:32857 every 8 connection(s)
I20260808 20:58:59.252887 3291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:58:59.254094 3381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:59.255664 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 3291
I20260808 20:58:59.256002 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:58:59.255894 3381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } has no permanent_uuid. Determining permanent_uuid...
I20260808 20:58:59.256242 16290 external_mini_cluster.cc:1428] Setting key edca53c4d6f1af3813f03c83521dc760
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.189@KRBTEST.COM; defaulting to no policy
I20260808 20:58:59.267129 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.256831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32857 (local address 127.0.0.1:44722)
0808 20:58:59.257067 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.257118 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:59.257691 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:59.258175 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:59.258178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:59.258498 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:59.258681 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.258686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.259856 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.259857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.260562 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.260565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.261076 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.261077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.261144 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.261749 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:59.261758 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:59.263297 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:59.265522 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.265529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.265532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.265740 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.266006 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.266007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.266007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.266138 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.266575 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:59.266582 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:59.266656 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 20:58:59.266757 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 20:58:59.266997 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":89,"thread_start_us":98,"threads_started":1}
I20260808 20:58:59.267132 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.256928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.15.232.190:32857)
0808 20:58:59.257128 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.257138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.257217 (+ 79us) server_negotiation.cc:407] Connection header received
0808 20:58:59.257979 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.257982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.258006 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.258134 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.258825 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.259759 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.260663 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.260985 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.261109 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.263406 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.263421 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.263421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.263428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.265405 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.265832 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.265835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.265836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.265932 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.266217 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.266218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.266218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.266472 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.266549 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.266796 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.266916 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.189@KRBTEST.COM" created.
I20260808 20:58:59.268173 3381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } has no permanent_uuid. Determining permanent_uuid...
I20260808 20:58:59.268568 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.268445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:45709 (local address 127.0.0.1:36072)
0808 20:58:59.268492 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.268544 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 20:58:59.268707 3310 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
I20260808 20:58:59.269153 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.269038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:45709 (local address 127.0.0.1:36074)
0808 20:58:59.269104 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.269129 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 20:58:59.269258 3381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.189:45709: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab.
Entry for principal kudu/127.15.232.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab.
I20260808 20:58:59.286840 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.189:45709
--webserver_interface=127.15.232.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab
--principal=kudu/127.15.232.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:32857,127.15.232.189:45709,127.15.232.188:40943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:58:59.318540 3381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } attempt: 1
I20260808 20:58:59.319180 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.319019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:45709 (local address 127.0.0.1:36076)
0808 20:58:59.319099 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.319146 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 20:58:59.319643 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.319524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:45709 (local address 127.0.0.1:36078)
0808 20:58:59.319579 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.319617 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 20:58:59.319743 3381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.189:45709: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
W20260808 20:58:59.376932 3394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:59.377204 3394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:59.377245 3394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:59.378423 3394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:58:59.378507 3394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:59.378533 3394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:59.378556 3394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:58:59.378583 3394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:58:59.380188 3394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:32857,127.15.232.189:45709,127.15.232.188:40943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab
--principal=kudu/127.15.232.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.189:45709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.232.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:59.380493 3394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:59.380756 3394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:59.386353 3404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:59.386390 3405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:59.386426 3409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:59.386963 3394 server_base.cc:1034] running on GCE node
I20260808 20:58:59.387250 3394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:59.387632 3394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:59.387892 3381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } attempt: 2
I20260808 20:58:59.388332 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.388229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:45709 (local address 127.0.0.1:36088)
0808 20:58:59.388274 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.388307 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 20:58:59.388631 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.388549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:45709 (local address 127.0.0.1:36092)
0808 20:58:59.388593 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.388616 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 20:58:59.388715 3381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.189:45709: Network error: Client connection negotiation failed: client connection to 127.15.232.189:45709: connect: Connection refused (error 111)
I20260808 20:58:59.388823 3394 hybrid_clock.cc:648] HybridClock initialized: now 1786222739388790 us; error 48 us; skew 500 ppm
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:59.391953 3394 init.cc:377] Logged in from keytab as kudu/127.15.232.189@KRBTEST.COM (short username kudu)
I20260808 20:58:59.392463 3394 webserver.cc:460] Webserver started at http://127.15.232.189:43079/ using document root <none> and password file <none>
I20260808 20:58:59.392758 3394 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:59.392839 3394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:59.393013 3394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:58:59.394304 3394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/data/instance:
uuid: "26e58b3bc4524b30b326c3517f58e919"
format_stamp: "Formatted at 2026-08-08 20:58:59 on dist-test-slave-c12x"
server_key: "30cf86d531adf73cc7ef9253750824d7"
server_key_iv: "db2daca3984a99728022d2076766d1b3"
server_key_version: "encryptionkey@0"
I20260808 20:58:59.394727 3394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/wal/instance:
uuid: "26e58b3bc4524b30b326c3517f58e919"
format_stamp: "Formatted at 2026-08-08 20:58:59 on dist-test-slave-c12x"
server_key: "30cf86d531adf73cc7ef9253750824d7"
server_key_iv: "db2daca3984a99728022d2076766d1b3"
server_key_version: "encryptionkey@0"
I20260808 20:58:59.396325 3394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 20:58:59.397209 3419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:59.397507 3394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:58:59.397629 3394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/wal
uuid: "26e58b3bc4524b30b326c3517f58e919"
format_stamp: "Formatted at 2026-08-08 20:58:59 on dist-test-slave-c12x"
server_key: "30cf86d531adf73cc7ef9253750824d7"
server_key_iv: "db2daca3984a99728022d2076766d1b3"
server_key_version: "encryptionkey@0"
I20260808 20:58:59.397716 3394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:59.423818 3394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:59.426344 3394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:59.426532 3394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:59.431113 3394 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.189:45709
I20260808 20:58:59.431159 3480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.189:45709 every 8 connection(s)
I20260808 20:58:59.431629 3394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
I20260808 20:58:59.432564 3481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:59.433816 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 3394
I20260808 20:58:59.433956 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-1/wal/instance
I20260808 20:58:59.434158 16290 external_mini_cluster.cc:1428] Setting key 1ae5acff1b87dd16edc5b8795f220efd
I20260808 20:58:59.434785 3481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.188@KRBTEST.COM; defaulting to no policy
I20260808 20:58:59.445307 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.435762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44734 (local address 127.15.232.190:32857)
0808 20:58:59.435834 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.435837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.436063 (+ 226us) server_negotiation.cc:407] Connection header received
0808 20:58:59.436606 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.436608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.436623 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.436677 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.437371 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.437874 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.438922 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.439145 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.439225 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.441975 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.441976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.441976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.441985 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.443818 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.444328 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.444330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.444330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.444406 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.444683 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.444684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.444684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.444811 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.444868 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.445154 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.445234 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 20:58:59.445516 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.435715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32857 (local address 127.0.0.1:44734)
0808 20:58:59.435915 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.435961 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:59.436485 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:59.436751 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:59.436755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:59.437043 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:59.437257 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.437262 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.437980 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.437981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.438805 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.438808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.439236 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.439237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.439382 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.440074 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:59.440085 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:59.441825 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:59.443933 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.443951 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.443953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.444239 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.444487 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.444488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.444488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.444614 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.444876 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:59.444880 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:59.445019 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 20:58:59.445125 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 20:58:59.445269 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":115,"threads_started":1}
I20260808 20:58:59.446276 3481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.188@KRBTEST.COM" created.
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
I20260808 20:58:59.456362 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.446636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:45709 (local address 127.0.0.1:36096)
0808 20:58:59.446711 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.446725 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:59.446781 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:59.447110 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:59.447111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:59.447264 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:59.447336 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.447338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.448654 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.448656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.449229 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.449231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.449917 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.449918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.449990 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.450558 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:59.450563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:59.452294 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:59.454608 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.454610 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.454610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.454859 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.455139 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.455140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.455141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.455225 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.455991 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:59.455993 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:59.456057 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 20:58:59.456147 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 20:58:59.456256 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 20:58:59.456547 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.446755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36096 (local address 127.15.232.189:45709)
0808 20:58:59.446935 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.446940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.446970 (+ 30us) server_negotiation.cc:407] Connection header received
0808 20:58:59.446995 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.446997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.447013 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.447096 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.447451 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.448551 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.449319 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.449814 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.449900 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.452414 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.452424 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.452425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.452433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.454507 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.454981 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.454985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.454986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.455060 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.455317 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.455319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.455319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.455759 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.456028 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.456283 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.456364 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
I20260808 20:58:59.457541 3481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } has no permanent_uuid. Determining permanent_uuid...
I20260808 20:58:59.457993 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.457848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51340)
0808 20:58:59.457916 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.457969 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 20:58:59.458170 3422 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
I20260808 20:58:59.458778 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.458623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51342)
0808 20:58:59.458729 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.458759 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260808 20:58:59.459046 3481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:40943: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab.
Entry for principal kudu/127.15.232.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab.
I20260808 20:58:59.467840 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.188:40943
--webserver_interface=127.15.232.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab
--principal=kudu/127.15.232.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:32857,127.15.232.189:45709,127.15.232.188:40943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:58:59.479882 3381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } attempt: 3
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
I20260808 20:58:59.488269 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.480381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:45709 (local address 127.0.0.1:36110)
0808 20:58:59.480476 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.480487 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:59.480562 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:59.480731 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:59.480732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:59.480928 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:59.481004 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.481006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.481733 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.481734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.482258 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.482260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.482712 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.482714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.482799 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.483350 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:59.483356 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:59.485034 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:59.487097 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.487098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.487098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.487272 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.487563 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.487564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.487564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.487628 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.487912 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:59.487914 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:59.487968 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 20:58:59.488037 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 20:58:59.488161 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 20:58:59.488269 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.480473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.15.232.189:45709)
0808 20:58:59.480546 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.480548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.480566 (+ 18us) server_negotiation.cc:407] Connection header received
0808 20:58:59.480634 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.480636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.480658 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.480726 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.481107 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.481639 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.482351 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.482630 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.482713 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.485156 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.485158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.485158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.485167 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.486920 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.487377 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.487380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.487380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.487484 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.487697 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.487699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.487699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.487835 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.487910 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.488079 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.488162 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 20:58:59.488993 3381 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } has no permanent_uuid. Determining permanent_uuid...
I20260808 20:58:59.489398 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.489292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51352)
0808 20:58:59.489343 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.489376 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 20:58:59.489766 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.489670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51358)
0808 20:58:59.489718 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.489743 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 20:58:59.489872 3381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:40943: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
I20260808 20:58:59.508324 3481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } attempt: 1
I20260808 20:58:59.508975 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.508789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51370)
0808 20:58:59.508890 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.508940 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 20:58:59.509438 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.509341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51384)
0808 20:58:59.509389 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.509417 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 20:58:59.509560 3481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:40943: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
I20260808 20:58:59.521077 3381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } attempt: 1
I20260808 20:58:59.521704 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.521528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51400)
0808 20:58:59.521626 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.521672 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 20:58:59.522096 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.521992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51416)
0808 20:58:59.522041 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.522075 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 20:58:59.522194 3381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:40943: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
W20260808 20:58:59.560592 3491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:58:59.560858 3491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:58:59.560897 3491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:58:59.562098 3491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:58:59.562177 3491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:58:59.562204 3491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:58:59.562233 3491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:58:59.562263 3491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:58:59.563966 3491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:32857,127.15.232.189:45709,127.15.232.188:40943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab
--principal=kudu/127.15.232.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.188:40943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.232.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:58:59.564273 3491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:58:59.564548 3491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:58:59.569936 3497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:59.569936 3501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:58:59.570824 3498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:58:59.571115 3491 server_base.cc:1034] running on GCE node
I20260808 20:58:59.571408 3491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:58:59.571779 3491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:58:59.572959 3491 hybrid_clock.cc:648] HybridClock initialized: now 1786222739572949 us; error 35 us; skew 500 ppm
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:58:59.575752 3491 init.cc:377] Logged in from keytab as kudu/127.15.232.188@KRBTEST.COM (short username kudu)
I20260808 20:58:59.576287 3491 webserver.cc:460] Webserver started at http://127.15.232.188:44635/ using document root <none> and password file <none>
I20260808 20:58:59.576511 3491 fs_manager.cc:359] Metadata directory not provided
I20260808 20:58:59.576555 3491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:58:59.576653 3491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:58:59.577732 3481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } attempt: 2
I20260808 20:58:59.577739 3491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/data/instance:
uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a"
format_stamp: "Formatted at 2026-08-08 20:58:59 on dist-test-slave-c12x"
server_key: "3b87bfb36949fc8e3070b79743cdd8c8"
server_key_iv: "90521cd02757407a6d546e4045d557cd"
server_key_version: "encryptionkey@0"
I20260808 20:58:59.578131 3491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/wal/instance:
uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a"
format_stamp: "Formatted at 2026-08-08 20:58:59 on dist-test-slave-c12x"
server_key: "3b87bfb36949fc8e3070b79743cdd8c8"
server_key_iv: "90521cd02757407a6d546e4045d557cd"
server_key_version: "encryptionkey@0"
I20260808 20:58:59.578183 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.578049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51418)
0808 20:58:59.578113 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.578153 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 20:58:59.578528 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.578432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51426)
0808 20:58:59.578479 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.578506 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 20:58:59.578622 3481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:40943: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
I20260808 20:58:59.579843 3491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 20:58:59.580833 3514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:58:59.581257 3491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:58:59.581435 3491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/wal
uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a"
format_stamp: "Formatted at 2026-08-08 20:58:59 on dist-test-slave-c12x"
server_key: "3b87bfb36949fc8e3070b79743cdd8c8"
server_key_iv: "90521cd02757407a6d546e4045d557cd"
server_key_version: "encryptionkey@0"
I20260808 20:58:59.581549 3491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:58:59.597357 3381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } attempt: 2
I20260808 20:58:59.597915 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.597759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51440)
0808 20:58:59.597830 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.597875 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 20:58:59.598302 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.598210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51452)
0808 20:58:59.598257 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.598284 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 20:58:59.598415 3381 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:40943: Network error: Client connection negotiation failed: client connection to 127.15.232.188:40943: connect: Connection refused (error 111)
I20260808 20:58:59.599040 3491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:58:59.601608 3491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:58:59.601733 3491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:58:59.605892 3491 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.188:40943
I20260808 20:58:59.605919 3592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.188:40943 every 8 connection(s)
I20260808 20:58:59.606331 3491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
I20260808 20:58:59.607290 3594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:58:59.609036 3594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } has no permanent_uuid. Determining permanent_uuid...
I20260808 20:58:59.614939 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 3491
I20260808 20:58:59.615087 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786222573557010-16290-0/minicluster-data/master-2/wal/instance
I20260808 20:58:59.615321 16290 external_mini_cluster.cc:1428] Setting key 11ad95994363d6a41a5a9dbd69e7f2e2
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:58:59.620268 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.609914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.15.232.190:32857)
0808 20:58:59.610042 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.610045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.610248 (+ 203us) server_negotiation.cc:407] Connection header received
0808 20:58:59.610777 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.610779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.610790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.610860 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.611606 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.612090 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.613195 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.613393 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.613470 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.616075 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.616077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.616077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.616087 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.618317 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.618891 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.618893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.618893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.619019 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.619337 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.619338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.619338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.619577 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.619675 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.620071 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.620171 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 20:58:59.620544 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.609870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32857 (local address 127.0.0.1:44746)
0808 20:58:59.610119 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.610157 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:59.610666 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:59.610935 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:59.610938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:59.611214 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:59.611505 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.611511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.612190 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.612191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.613099 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.613102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.613490 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.613492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.613622 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.614175 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:59.614185 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:59.615951 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:59.618428 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.618436 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.618439 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.618815 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.619101 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.619102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.619103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.619269 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.619665 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:59.619673 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:59.619890 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 20:58:59.620044 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 20:58:59.620215 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":114,"threads_started":1}
I20260808 20:58:59.621168 3594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
I20260808 20:58:59.627723 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.615893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.15.232.188:40943)
0808 20:58:59.616368 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.616374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.616399 (+ 25us) server_negotiation.cc:407] Connection header received
0808 20:58:59.616424 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.616426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.616442 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.616535 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.617329 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.618724 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.619850 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.620308 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.622438 (+ 2130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.623791 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.623803 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.623803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.623812 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.625903 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.626654 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.626658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.626658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.626733 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.627004 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.627005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.627006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.627217 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.627409 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.627493 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.627579 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":116,"threads_started":1}
I20260808 20:58:59.631402 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.621931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36118 (local address 127.15.232.189:45709)
0808 20:58:59.622392 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.622394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.622405 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:58:59.622432 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.622433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.622445 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.622509 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.622899 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.623723 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.624990 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.625373 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.625584 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.628014 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.628016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.628016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.628023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.629783 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.630191 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.630192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.630192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.630264 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.630637 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.630638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.630638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.630965 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.631045 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.631256 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.631333 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431}
I20260808 20:58:59.631537 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.621810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:45709 (local address 127.0.0.1:36118)
0808 20:58:59.621861 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.621875 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:59.621949 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:59.622559 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:59.622560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:59.622722 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:59.622789 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.622791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.623801 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.623802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.624620 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.624624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.625680 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.625682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.625802 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.626271 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:59.626276 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:59.627931 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:59.629870 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.629871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.629872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.630098 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.630469 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.630470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.630471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.630531 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.631065 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:59.631066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:59.631108 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 20:58:59.631224 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 20:58:59.631330 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 20:58:59.632489 3594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
I20260808 20:58:59.638945 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.628721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.15.232.190:32857)
0808 20:58:59.628762 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.628764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.628772 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:58:59.628798 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.628799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.628812 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.628880 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.629474 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.630080 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.631220 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.631574 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.631652 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.635789 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.635791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.635791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.635799 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.637436 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.637916 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.637918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.637918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.637997 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.638258 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.638260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.638260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.638425 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.638640 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.638719 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.638814 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 20:58:59.642184 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.632816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51484)
0808 20:58:59.632894 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.632911 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:59.632970 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:59.633163 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:59.633164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:59.633525 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:59.633588 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.633657 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.634430 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.634431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.636167 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.636171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.636601 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.636601 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.636651 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.637233 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:59.637237 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:59.638833 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:59.640950 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.640951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.640951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.641131 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.641396 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.641397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.641397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.641499 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.641801 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:59.641802 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:59.641851 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 20:58:59.641927 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 20:58:59.642043 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 20:58:59.642190 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.632962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.15.232.188:40943)
0808 20:58:59.633005 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.633007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.633016 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:58:59.633038 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.633039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.633050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.633115 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.633888 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.634352 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.636260 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.636521 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.636604 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.638935 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.638936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.638937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.638943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.640870 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.641210 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.641211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.641212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.641295 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.641611 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.641611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.641612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.641731 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.641785 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.641954 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.642103 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":24}
I20260808 20:58:59.644222 3594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a: Bootstrap starting.
I20260808 20:58:59.645413 3594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:59.645743 3594 log.cc:824] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a: Log is configured to *not* fsync() on all Append() calls
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
I20260808 20:58:59.646766 3594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a: No bootstrap required, opened a new log
I20260808 20:58:59.649367 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.640635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36124 (local address 127.15.232.189:45709)
0808 20:58:59.640705 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.640707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.640757 (+ 50us) server_negotiation.cc:407] Connection header received
0808 20:58:59.640839 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.640840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.640849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.640893 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.641497 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.642254 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.643223 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.643412 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.643554 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.646320 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.646321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.646322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.646331 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.647981 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.648399 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.648400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.648400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.648455 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.648827 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.648828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.648829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.648975 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.649136 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.649233 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.649302 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 20:58:59.649331 3594 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } }
I20260808 20:58:59.649534 3594 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:59.649621 3594 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc3ed8ecf36b4aa4afb4bf4660dca68a, State: Initialized, Role: FOLLOWER
I20260808 20:58:59.649761 3594 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } }
I20260808 20:58:59.650054 3601 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.650125 3594 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:58:59.650133 3601 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:58:59.652138 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 20:58:59.652877 3613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 20:58:59.652930 3613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 20:58:59.653151 3491 master_runner.cc:186] Error getting master registration for 127.15.232.190:32857: OK, instance_id { permanent_uuid: "ea38efa1470a4ce684765299b763f451" instance_seqno: 1786222739250176 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 20:58:59.656158 3491 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 20:58:59.662896 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.653897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.15.232.188:40943)
0808 20:58:59.654117 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.654119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.655142 (+ 1023us) server_negotiation.cc:407] Connection header received
0808 20:58:59.655242 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.655243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.655253 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.655285 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.656034 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.656707 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.657646 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.657964 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.658043 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.659896 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.659898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.659899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.659906 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.661504 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.661903 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.661904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.661904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.661979 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.662212 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.662213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.662214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.662395 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.662483 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.662703 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.662792 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260808 20:58:59.663748 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.654161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.15.232.189:45709)
0808 20:58:59.654203 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.654205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.654372 (+ 167us) server_negotiation.cc:407] Connection header received
0808 20:58:59.654486 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.654488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.654498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.654526 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.655388 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.656182 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.657293 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.657599 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.657640 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.659504 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.659506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.659506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.659515 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.662113 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.662534 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.662535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.662536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.662597 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.662809 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.662810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.662810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.662952 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.663025 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.663558 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.663651 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 20:58:59.666019 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.654238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.15.232.190:32857)
0808 20:58:59.654311 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.654313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.654324 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:58:59.654352 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.654353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.654364 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.654413 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.657990 (+ 3577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.658608 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.660611 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.660966 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.661015 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.662854 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.662855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.662855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.662863 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.664874 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.665191 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.665192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.665192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.665257 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.665469 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.665470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.665470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.665591 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.665699 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.665834 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.665917 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 20:58:59.669795 3481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } attempt: 3
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
I20260808 20:58:59.678043 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.670168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51494)
0808 20:58:59.670236 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.670249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:59.670303 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:59.670432 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:59.670433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:59.670634 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:59.670683 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.670685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.671316 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.671317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.672144 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.672147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.672622 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.672623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.672682 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.673207 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:59.673213 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:59.674756 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:59.676799 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.676800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.676801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.676983 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.677248 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.677249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.677250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.677368 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.677735 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:59.677736 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:59.677786 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 20:58:59.677865 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 20:58:59.677969 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 20:58:59.678097 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.670251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51494 (local address 127.15.232.188:40943)
0808 20:58:59.670305 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.670307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.670317 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:58:59.670371 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.670372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.670379 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.670426 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.670750 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.671233 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.672247 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.672543 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.672614 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.675024 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.675026 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.675026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.675034 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.676698 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.677064 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.677065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.677066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.677154 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.677456 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.677457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.677458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.677606 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.677680 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.677907 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.678001 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 20:58:59.680478 3481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919: Bootstrap starting.
I20260808 20:58:59.681296 3481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:59.681617 3481 log.cc:824] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:59.682526 3481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919: No bootstrap required, opened a new log
I20260808 20:58:59.684623 3481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } }
I20260808 20:58:59.684787 3481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:59.684825 3481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e58b3bc4524b30b326c3517f58e919, State: Initialized, Role: FOLLOWER
I20260808 20:58:59.684944 3481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } }
I20260808 20:58:59.685272 3626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.685393 3626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:58:59.685340 3481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 20:58:59.688376 3637 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 20:58:59.688506 3637 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 20:58:59.687786 3394 master_runner.cc:186] Error getting master registration for 127.15.232.190:32857: OK, instance_id { permanent_uuid: "ea38efa1470a4ce684765299b763f451" instance_seqno: 1786222739250176 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 20:58:59.689862 3394 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 20:58:59.697610 3381 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } attempt: 3
Aug 08 20:58:59 dist-test-slave-c12x krb5kdc[3274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222739, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
I20260808 20:58:59.705658 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.698151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:40943 (local address 127.0.0.1:51500)
0808 20:58:59.698269 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 20:58:59.698283 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:58:59.698325 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:58:59.698718 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:58:59.698719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:58:59.698901 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:58:59.698946 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.698948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.699681 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.699682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.700239 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:58:59.700242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.700679 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.700681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:58:59.700744 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.701378 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:58:59.701384 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:58:59.702726 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:58:59.704494 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.704495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.704496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.704735 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.704945 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:58:59.704946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:58:59.704946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:58:59.705014 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:58:59.705299 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:58:59.705301 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:58:59.705350 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 20:58:59.705440 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 20:58:59.705548 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260808 20:58:59.705650 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:58:59.698488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51500 (local address 127.15.232.188:40943)
0808 20:58:59.698542 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 20:58:59.698545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:58:59.698553 (+ 8us) server_negotiation.cc:407] Connection header received
0808 20:58:59.698577 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:58:59.698578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:58:59.698588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:58:59.698638 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:58:59.699038 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.699594 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.700334 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:58:59.700528 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:58:59.700657 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:58:59.702825 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:58:59.702826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:58:59.702827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:58:59.702836 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:58:59.704374 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.704798 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.704799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.704800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.704869 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:58:59.705092 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:58:59.705093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:58:59.705094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:58:59.705221 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:58:59.705295 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 20:58:59.705467 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 20:58:59.705548 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:58:59.708033 3381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451: Bootstrap starting.
I20260808 20:58:59.708982 3381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451: Neither blocks nor log segments found. Creating new log.
I20260808 20:58:59.709347 3381 log.cc:824] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451: Log is configured to *not* fsync() on all Append() calls
I20260808 20:58:59.710178 3381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451: No bootstrap required, opened a new log
I20260808 20:58:59.712055 3381 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } }
I20260808 20:58:59.712208 3381 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:58:59.712252 3381 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea38efa1470a4ce684765299b763f451, State: Initialized, Role: FOLLOWER
I20260808 20:58:59.712381 3381 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } }
I20260808 20:58:59.712764 3381 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:58:59.712765 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.712989 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 20:58:59.714706 3655 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 20:58:59.714813 3655 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 20:58:59.717324 3291 master_runner.cc:263] No leader master found from master ea38efa1470a4ce684765299b763f451
I20260808 20:58:59.717374 3291 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 20:58:59.746537 3316 tablet.cc:2369] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:58:59.774856 3640 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:58:59.774996 3640 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } }
I20260808 20:58:59.775249 3640 leader_election.cc:288] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26e58b3bc4524b30b326c3517f58e919 (127.15.232.189:45709), cc3ed8ecf36b4aa4afb4bf4660dca68a (127.15.232.188:40943)
I20260808 20:58:59.775717 3451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea38efa1470a4ce684765299b763f451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26e58b3bc4524b30b326c3517f58e919" is_pre_election: true
I20260808 20:58:59.775888 3451 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea38efa1470a4ce684765299b763f451 in term 0.
I20260808 20:58:59.776074 3554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea38efa1470a4ce684765299b763f451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" is_pre_election: true
I20260808 20:58:59.776243 3554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea38efa1470a4ce684765299b763f451 in term 0.
I20260808 20:58:59.776471 3310 leader_election.cc:302] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26e58b3bc4524b30b326c3517f58e919, ea38efa1470a4ce684765299b763f451; no voters:
I20260808 20:58:59.776733 3640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:58:59.776803 3640 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:58:59.776834 3640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:59.777731 3640 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } }
I20260808 20:58:59.777884 3640 leader_election.cc:288] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [CANDIDATE]: Term 1 election: Requested vote from peers 26e58b3bc4524b30b326c3517f58e919 (127.15.232.189:45709), cc3ed8ecf36b4aa4afb4bf4660dca68a (127.15.232.188:40943)
I20260808 20:58:59.778180 3451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea38efa1470a4ce684765299b763f451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26e58b3bc4524b30b326c3517f58e919"
I20260808 20:58:59.778278 3451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:59.779163 3451 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea38efa1470a4ce684765299b763f451 in term 1.
I20260808 20:58:59.779343 3310 leader_election.cc:302] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26e58b3bc4524b30b326c3517f58e919, ea38efa1470a4ce684765299b763f451; no voters:
I20260808 20:58:59.779477 3554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea38efa1470a4ce684765299b763f451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a"
I20260808 20:58:59.779585 3554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:58:59.780524 3554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea38efa1470a4ce684765299b763f451 in term 1.
I20260808 20:58:59.780756 3640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:58:59.780913 3640 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [term 1 LEADER]: Becoming Leader. State: Replica: ea38efa1470a4ce684765299b763f451, State: Running, Role: LEADER
I20260808 20:58:59.781023 3640 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } }
I20260808 20:58:59.781442 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea38efa1470a4ce684765299b763f451. Latest consensus state: current_term: 1 leader_uuid: "ea38efa1470a4ce684765299b763f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.781535 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [sys.catalog]: This master's current role is: LEADER
I20260808 20:58:59.781775 3662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:58:59.781961 3662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:58:59.784053 3554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [term 1 FOLLOWER]: Refusing update from remote peer ea38efa1470a4ce684765299b763f451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:58:59.784374 3640 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:58:59.786053 3601 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea38efa1470a4ce684765299b763f451. Latest consensus state: current_term: 1 leader_uuid: "ea38efa1470a4ce684765299b763f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.786191 3601 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:58:59.786603 3451 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [term 1 FOLLOWER]: Refusing update from remote peer ea38efa1470a4ce684765299b763f451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:58:59.786885 3640 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:58:59.788094 3626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea38efa1470a4ce684765299b763f451. Latest consensus state: current_term: 1 leader_uuid: "ea38efa1470a4ce684765299b763f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.788192 3626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:58:59.791244 3663 mvcc.cc:205] Tried to move back new op lower bound from 7316368342152749056 to 7316368342143791104. Current Snapshot: MvccSnapshot[applied={T|T < 7316368342152749056}]
I20260808 20:58:59.791596 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea38efa1470a4ce684765299b763f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.791692 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [sys.catalog]: This master's current role is: LEADER
I20260808 20:58:59.792164 3626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea38efa1470a4ce684765299b763f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.792260 3626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:58:59.792474 3670 mvcc.cc:205] Tried to move back new op lower bound from 7316368342152749056 to 7316368342143791104. Current Snapshot: MvccSnapshot[applied={T|T < 7316368342152749056}]
I20260808 20:58:59.792960 3640 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea38efa1470a4ce684765299b763f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.793051 3640 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451 [sys.catalog]: This master's current role is: LEADER
I20260808 20:58:59.793514 3601 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea38efa1470a4ce684765299b763f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea38efa1470a4ce684765299b763f451" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32857 } } peers { permanent_uuid: "26e58b3bc4524b30b326c3517f58e919" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 45709 } } peers { permanent_uuid: "cc3ed8ecf36b4aa4afb4bf4660dca68a" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 40943 } } }
I20260808 20:58:59.793624 3601 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 20:58:59.793941 3665 mvcc.cc:205] Tried to move back new op lower bound from 7316368342152749056 to 7316368342143791104. Current Snapshot: MvccSnapshot[applied={T|T < 7316368342152749056}]
I20260808 20:58:59.795233 3662 catalog_manager.cc:1347] Generated new cluster ID: 45417668243044c19af2977edcf749f7
I20260808 20:58:59.795292 3662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:58:59.843326 3662 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:58:59.844129 3662 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:58:59.860987 3662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea38efa1470a4ce684765299b763f451: Generated new TSK 0
I20260808 20:58:59.861222 3662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 20:59:00.653415 3613 catalog_manager.cc:1259] Loaded cluster ID: 45417668243044c19af2977edcf749f7
I20260808 20:59:00.653574 3613 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a: loading cluster ID for follower catalog manager: success
I20260808 20:59:00.654541 3613 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a: acquiring CA information for follower catalog manager: success
I20260808 20:59:00.654884 3613 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cc3ed8ecf36b4aa4afb4bf4660dca68a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 20:59:00.689113 3637 catalog_manager.cc:1259] Loaded cluster ID: 45417668243044c19af2977edcf749f7
I20260808 20:59:00.689203 3637 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919: loading cluster ID for follower catalog manager: success
I20260808 20:59:00.690224 3637 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919: acquiring CA information for follower catalog manager: success
I20260808 20:59:00.690630 3637 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 26e58b3bc4524b30b326c3517f58e919: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 20:59:00.930155 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 3291
I20260808 20:59:00.943636 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 3394
I20260808 20:59:00.956664 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 3491
2026-08-08T20:59:00Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3236 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:59:01 dist-test-slave-c12x krb5kdc[3684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:59:01 dist-test-slave-c12x krb5kdc[3684](info): set up 2 sockets
Aug 08 20:59:01 dist-test-slave-c12x krb5kdc[3684](info): commencing operation
krb5kdc: starting...
W20260808 20:59:02.257887 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:59:02 dist-test-slave-c12x krb5kdc[3684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:59:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:59:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:59:02.390534 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33237
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:41951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:33237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:02.477752 3701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:02.478016 3701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:02.478067 3701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:02.479244 3701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:59:02.479324 3701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:02.479350 3701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:02.479380 3701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:59:02.479410 3701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:59:02.480993 3701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:41951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:33237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:02.481282 3701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:02.481530 3701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:02.486908 3714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:02.486958 3712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:02.486995 3701 server_base.cc:1034] running on GCE node
W20260808 20:59:02.486953 3709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:02.487418 3701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:02.487860 3701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:02.489044 3701 hybrid_clock.cc:648] HybridClock initialized: now 1786222742489020 us; error 50 us; skew 500 ppm
Aug 08 20:59:02 dist-test-slave-c12x krb5kdc[3684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222742, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:02.491895 3701 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:59:02.492427 3701 webserver.cc:460] Webserver started at http://127.15.232.190:35417/ using document root <none> and password file <none>
I20260808 20:59:02.492661 3701 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:02.492710 3701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:02.492811 3701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:59:02.493948 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "cace7e063528439890388d03c8fa9e42"
format_stamp: "Formatted at 2026-08-08 20:59:02 on dist-test-slave-c12x"
server_key: "7a6d68d0705b3f38012f9311835c9c8e"
server_key_iv: "62a2a993bc6991080451b2a1d7210fec"
server_key_version: "encryptionkey@0"
I20260808 20:59:02.494298 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "cace7e063528439890388d03c8fa9e42"
format_stamp: "Formatted at 2026-08-08 20:59:02 on dist-test-slave-c12x"
server_key: "7a6d68d0705b3f38012f9311835c9c8e"
server_key_iv: "62a2a993bc6991080451b2a1d7210fec"
server_key_version: "encryptionkey@0"
I20260808 20:59:02.495878 3701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 20:59:02.496984 3722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:02.497186 3701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:59:02.497294 3701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "cace7e063528439890388d03c8fa9e42"
format_stamp: "Formatted at 2026-08-08 20:59:02 on dist-test-slave-c12x"
server_key: "7a6d68d0705b3f38012f9311835c9c8e"
server_key_iv: "62a2a993bc6991080451b2a1d7210fec"
server_key_version: "encryptionkey@0"
I20260808 20:59:02.497402 3701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:02.508503 3701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:02.510725 3701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:02.510851 3701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:02.516413 3701 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:33237
I20260808 20:59:02.516474 3794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:33237 every 8 connection(s)
I20260808 20:59:02.516875 3701 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:59:02.517305 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 3701
I20260808 20:59:02.517443 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:59:02.517685 16290 external_mini_cluster.cc:1428] Setting key 504742fa5a7115122b05b93ba976b6a4
I20260808 20:59:02.518117 3795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:02.521823 3795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42: Bootstrap starting.
I20260808 20:59:02.522573 3795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:02.522909 3795 log.cc:824] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:02.523916 3795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42: No bootstrap required, opened a new log
Aug 08 20:59:02 dist-test-slave-c12x krb5kdc[3684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:02.526270 3795 raft_consensus.cc:348] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cace7e063528439890388d03c8fa9e42" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33237 } }
I20260808 20:59:02.526494 3795 raft_consensus.cc:374] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:02.526537 3795 raft_consensus.cc:729] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cace7e063528439890388d03c8fa9e42, State: Initialized, Role: FOLLOWER
I20260808 20:59:02.526684 3795 consensus_queue.cc:260] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cace7e063528439890388d03c8fa9e42" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33237 } }
I20260808 20:59:02.526819 3795 raft_consensus.cc:388] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:59:02.526847 3795 raft_consensus.cc:482] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:59:02.526916 3795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:02.527762 3795 raft_consensus.cc:504] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cace7e063528439890388d03c8fa9e42" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33237 } }
I20260808 20:59:02.527922 3795 leader_election.cc:302] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cace7e063528439890388d03c8fa9e42; no voters:
I20260808 20:59:02.528228 3795 leader_election.cc:288] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:59:02.528270 3801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:59:02.528409 3801 raft_consensus.cc:686] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [term 1 LEADER]: Becoming Leader. State: Replica: cace7e063528439890388d03c8fa9e42, State: Running, Role: LEADER
I20260808 20:59:02.528510 3801 consensus_queue.cc:237] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cace7e063528439890388d03c8fa9e42" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33237 } }
I20260808 20:59:02.528604 3795 sys_catalog.cc:564] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:59:02.528877 3801 sys_catalog.cc:455] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cace7e063528439890388d03c8fa9e42. Latest consensus state: current_term: 1 leader_uuid: "cace7e063528439890388d03c8fa9e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cace7e063528439890388d03c8fa9e42" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33237 } } }
I20260808 20:59:02.528981 3801 sys_catalog.cc:458] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:02.528893 3802 sys_catalog.cc:455] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cace7e063528439890388d03c8fa9e42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cace7e063528439890388d03c8fa9e42" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33237 } } }
I20260808 20:59:02.529184 3802 sys_catalog.cc:458] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42 [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:02.530457 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:02.518328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41210 (local address 127.15.232.190:33237)
0808 20:59:02.518516 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:02.518526 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:02.518561 (+ 35us) server_negotiation.cc:407] Connection header received
0808 20:59:02.519350 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:02.519358 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:02.519442 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:02.519675 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:02.520461 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:02.521369 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:02.522472 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:02.523068 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:02.523196 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:02.526033 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:02.526037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:02.526037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:02.526051 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:02.528490 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:02.528980 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:02.528983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:02.528984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:02.529110 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:02.529422 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:02.529423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:02.529423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:02.529686 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:02.529872 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:02.530053 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 20:59:02.530206 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
W20260808 20:59:02.530925 3824 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 20:59:02.531015 3824 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 20:59:02.531090 3807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:59:02.531316 3807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:59:02.531771 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 20:59:02.533175 3807 catalog_manager.cc:1347] Generated new cluster ID: de7c2d17cbf9448688c03e7bac5e110c
I20260808 20:59:02.533231 3807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:59:02.553774 3807 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:59:02.554369 3807 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:59:02.559823 3807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cace7e063528439890388d03c8fa9e42: Generated new TSK 0
I20260808 20:59:02.560053 3807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 20:59:02.586179 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 3701
2026-08-08T20:59:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1628 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:59:02 dist-test-slave-c12x krb5kdc[3834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:59:02 dist-test-slave-c12x krb5kdc[3834](info): set up 2 sockets
Aug 08 20:59:02 dist-test-slave-c12x krb5kdc[3834](info): commencing operation
krb5kdc: starting...
W20260808 20:59:03.816344 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:59:03 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:59:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:59:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:59:03.954571 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:38411
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:38411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:04.040302 3852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:04.040597 3852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:04.040645 3852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:04.041891 3852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:59:04.041966 3852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:04.041991 3852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:04.042019 3852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:59:04.042052 3852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:59:04.043676 3852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:38411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:38411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:04.043972 3852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:04.044220 3852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:04.049571 3858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:04.049571 3862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:04.049599 3860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:04.050012 3852 server_base.cc:1034] running on GCE node
I20260808 20:59:04.050279 3852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:04.050755 3852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:04.051944 3852 hybrid_clock.cc:648] HybridClock initialized: now 1786222744051928 us; error 29 us; skew 500 ppm
Aug 08 20:59:04 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:04.054869 3852 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:59:04.055415 3852 webserver.cc:460] Webserver started at http://127.15.232.190:33249/ using document root <none> and password file <none>
I20260808 20:59:04.055689 3852 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:04.055737 3852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:04.055832 3852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:59:04.056917 3852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "db57001c48ac4b67891f56b4db0ca97a"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "20e8ddc02769ae6a25834668f2ee90a1"
server_key_iv: "cd093e8822e8a3944a68d854fb50993c"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.057253 3852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "db57001c48ac4b67891f56b4db0ca97a"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "20e8ddc02769ae6a25834668f2ee90a1"
server_key_iv: "cd093e8822e8a3944a68d854fb50993c"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.058588 3852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:59:04.059356 3873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.059574 3852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:59:04.059680 3852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "db57001c48ac4b67891f56b4db0ca97a"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "20e8ddc02769ae6a25834668f2ee90a1"
server_key_iv: "cd093e8822e8a3944a68d854fb50993c"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.059762 3852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:04.081674 3852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:04.083839 3852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:04.084015 3852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:04.088483 3852 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:38411
I20260808 20:59:04.088522 3939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:38411 every 8 connection(s)
I20260808 20:59:04.088887 3852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:59:04.089900 3940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:04.091369 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 3852
I20260808 20:59:04.091565 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:59:04.091813 16290 external_mini_cluster.cc:1428] Setting key 0ac2f7ea0d4384400fa96c42d8c4ba8b
I20260808 20:59:04.092945 3940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Bootstrap starting.
I20260808 20:59:04.093719 3940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:04.094038 3940 log.cc:824] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:04.094794 3940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: No bootstrap required, opened a new log
I20260808 20:59:04.096719 3940 raft_consensus.cc:348] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:04.096905 3940 raft_consensus.cc:374] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:04.096961 3940 raft_consensus.cc:729] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db57001c48ac4b67891f56b4db0ca97a, State: Initialized, Role: FOLLOWER
I20260808 20:59:04.097116 3940 consensus_queue.cc:260] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:04.097242 3940 raft_consensus.cc:388] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:59:04.097285 3940 raft_consensus.cc:482] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:59:04.097337 3940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:04.098187 3940 raft_consensus.cc:504] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:04.098315 3940 leader_election.cc:302] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db57001c48ac4b67891f56b4db0ca97a; no voters:
I20260808 20:59:04.098500 3940 leader_election.cc:288] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:59:04.098557 3952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:59:04.098709 3952 raft_consensus.cc:686] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 1 LEADER]: Becoming Leader. State: Replica: db57001c48ac4b67891f56b4db0ca97a, State: Running, Role: LEADER
I20260808 20:59:04.098788 3940 sys_catalog.cc:564] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:59:04.098840 3952 consensus_queue.cc:237] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
Aug 08 20:59:04 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:04.099393 3954 sys_catalog.cc:455] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: SysCatalogTable state changed. Reason: New leader db57001c48ac4b67891f56b4db0ca97a. Latest consensus state: current_term: 1 leader_uuid: "db57001c48ac4b67891f56b4db0ca97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } } }
I20260808 20:59:04.099498 3954 sys_catalog.cc:458] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:04.099777 3953 sys_catalog.cc:455] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db57001c48ac4b67891f56b4db0ca97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } } }
I20260808 20:59:04.099845 3953 sys_catalog.cc:458] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:04.100425 3962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:59:04.101019 3962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:59:04.102885 3962 catalog_manager.cc:1347] Generated new cluster ID: fa129c8b739d4574b543303e9980170b
I20260808 20:59:04.102988 3962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:59:04.104166 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.092418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.15.232.190:38411)
0808 20:59:04.092660 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.092671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.092704 (+ 33us) server_negotiation.cc:407] Connection header received
0808 20:59:04.093243 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.093252 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.093324 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.093487 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:04.094179 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.095182 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.096029 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.096544 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.096668 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.099770 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:04.099775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:04.099775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:04.099789 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:04.101937 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.102505 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.102508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.102508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.102630 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.102865 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.102866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.102866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.103215 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:04.103359 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.103817 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.103959 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":101,"threads_started":1}
I20260808 20:59:04.119489 3962 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:59:04.120316 3962 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:59:04.127260 3962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Generated new TSK 0
I20260808 20:59:04.127518 3962 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:59:04.156446 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:38411
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:04.245180 3977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:04.245409 3977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:04.245436 3977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:04.246595 3977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:04.246632 3977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:04.246660 3977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:59:04.248224 3977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:04.248441 3977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:04.248639 3977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:04.254488 3985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:04.254508 3991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:04.254500 3986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:04.254808 3977 server_base.cc:1034] running on GCE node
I20260808 20:59:04.255046 3977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:04.255290 3977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:04.256485 3977 hybrid_clock.cc:648] HybridClock initialized: now 1786222744256433 us; error 41 us; skew 500 ppm
Aug 08 20:59:04 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:04.264995 3977 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:59:04.265513 3977 webserver.cc:460] Webserver started at http://127.15.232.129:36759/ using document root <none> and password file <none>
I20260808 20:59:04.265782 3977 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:04.265870 3977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:04.266050 3977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:59:04.267231 3977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "852838d44a2e4a838b2d5c1c2a3c5db5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "c449d90c5362de319eb05d4a9a232bc9"
server_key_iv: "6151816ae314024abb60ba001379ad19"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.267688 3977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "852838d44a2e4a838b2d5c1c2a3c5db5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "c449d90c5362de319eb05d4a9a232bc9"
server_key_iv: "6151816ae314024abb60ba001379ad19"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.269170 3977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:04.270042 4002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.270284 3977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:04.270442 3977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "852838d44a2e4a838b2d5c1c2a3c5db5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "c449d90c5362de319eb05d4a9a232bc9"
server_key_iv: "6151816ae314024abb60ba001379ad19"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.270533 3977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:04.314380 3977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:04.316569 3977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:04.316764 3977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:04.317034 3977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:04.317392 3977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:04.317448 3977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.317492 3977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:04.317520 3977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.323359 3977 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:35965
I20260808 20:59:04.323487 4165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:35965 every 8 connection(s)
I20260808 20:59:04.323824 3977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
Aug 08 20:59:04 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:04.333566 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 3977
I20260808 20:59:04.333696 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:59:04.333941 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.324644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:53161 (local address 127.15.232.190:38411)
0808 20:59:04.324689 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.324691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.324923 (+ 232us) server_negotiation.cc:407] Connection header received
0808 20:59:04.325384 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.325385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.325397 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.325474 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:04.326482 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.327006 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.328315 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.328557 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.328624 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.330905 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:04.330907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:04.330907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:04.330914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:04.332452 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.332891 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.332891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.332892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.332947 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.333169 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.333169 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.333169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.333280 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:04.333349 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.333784 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.333872 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 20:59:04.333948 16290 external_mini_cluster.cc:1428] Setting key ee63f3267948f41bb49a7760b00901e3
I20260808 20:59:04.334167 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.324587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:53161)
0808 20:59:04.324756 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.324800 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:04.325295 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:04.325529 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:04.325532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:04.326126 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:04.326363 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.326369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.327136 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.327138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.328086 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.328089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.328672 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.328673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.328794 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.329331 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:04.329339 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:04.330804 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:04.332543 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:04.332551 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:04.332553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:04.332809 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:04.333008 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:04.333008 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:04.333009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:04.333113 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:04.333336 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:04.333367 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:04.333553 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 20:59:04.333718 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 20:59:04.333914 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260808 20:59:04.334913 4167 heartbeater.cc:347] Connected to a master server at 127.15.232.190:38411
I20260808 20:59:04.335048 4167 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:04.335250 4167 heartbeater.cc:511] Master 127.15.232.190:38411 requested a full tablet report, sending...
I20260808 20:59:04.335794 3895 ts_manager.cc:194] Registered new tserver with Master: 852838d44a2e4a838b2d5c1c2a3c5db5 (127.15.232.129:35965)
I20260808 20:59:04.336436 3895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:53161
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:59:04.362154 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:38411
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:04.447960 4173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:04.448207 4173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:04.448251 4173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:04.449501 4173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:04.449577 4173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:04.449626 4173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:59:04.451283 4173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:04.451577 4173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:04.451815 4173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:04.457168 4182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:04.457417 4184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:04.457330 4173 server_base.cc:1034] running on GCE node
W20260808 20:59:04.457216 4181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:04.457742 4173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:04.457990 4173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:04.459213 4173 hybrid_clock.cc:648] HybridClock initialized: now 1786222744459193 us; error 34 us; skew 500 ppm
Aug 08 20:59:04 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:04.461999 4173 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:59:04.462541 4173 webserver.cc:460] Webserver started at http://127.15.232.130:42449/ using document root <none> and password file <none>
I20260808 20:59:04.462793 4173 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:04.462865 4173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:04.463009 4173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:59:04.464280 4173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "95181f5851114da09ff09791d505abd5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "148f2775c79a49cdcb288f7cd5a273c5"
server_key_iv: "27a322db7a7d82aaf38c16629adca2c1"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.464675 4173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "95181f5851114da09ff09791d505abd5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "148f2775c79a49cdcb288f7cd5a273c5"
server_key_iv: "27a322db7a7d82aaf38c16629adca2c1"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.466095 4173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 20:59:04.466938 4196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.467192 4173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:59:04.467298 4173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "95181f5851114da09ff09791d505abd5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "148f2775c79a49cdcb288f7cd5a273c5"
server_key_iv: "27a322db7a7d82aaf38c16629adca2c1"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.467360 4173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:04.494417 4173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:04.496778 4173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:04.496896 4173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:04.497097 4173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:04.497469 4173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:04.497507 4173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.497556 4173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:04.497584 4173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.503751 4173 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:32785
I20260808 20:59:04.503840 4357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:32785 every 8 connection(s)
I20260808 20:59:04.504215 4173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:59:04.508996 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 4173
I20260808 20:59:04.509239 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:59:04.509471 16290 external_mini_cluster.cc:1428] Setting key 3ea50d5fedb063e7e102a556ff8859ef
Aug 08 20:59:04 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:04.514264 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.505061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:42167 (local address 127.15.232.190:38411)
0808 20:59:04.505113 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.505115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.505355 (+ 240us) server_negotiation.cc:407] Connection header received
0808 20:59:04.505905 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.505907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.505920 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.505976 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:04.507003 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.507580 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.508615 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.508817 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.508895 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.511290 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:04.511292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:04.511292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:04.511300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:04.512805 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.513257 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.513259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.513259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.513319 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.513586 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.513587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.513587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.513709 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:04.513784 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.514079 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.514166 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 20:59:04.514478 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.505042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:42167)
0808 20:59:04.505225 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.505262 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:04.505790 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:04.506050 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:04.506054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:04.506671 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:04.506877 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.506882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.507675 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.507676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.508494 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.508497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.508903 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.508905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.509036 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.509669 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:04.509680 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:04.511182 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:04.512912 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:04.512920 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:04.512923 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:04.513167 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:04.513397 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:04.513398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:04.513398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:04.513515 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:04.513788 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:04.513796 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:04.513941 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 20:59:04.514054 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 20:59:04.514232 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260808 20:59:04.515126 4359 heartbeater.cc:347] Connected to a master server at 127.15.232.190:38411
I20260808 20:59:04.515224 4359 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:04.515426 4359 heartbeater.cc:511] Master 127.15.232.190:38411 requested a full tablet report, sending...
I20260808 20:59:04.515910 3895 ts_manager.cc:194] Registered new tserver with Master: 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785)
I20260808 20:59:04.516304 3895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:42167
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:59:04.537591 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:38411
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:04.624388 4368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:04.624644 4368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:04.624684 4368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:04.625859 4368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:04.625936 4368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:04.625982 4368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:59:04.627694 4368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:04.627986 4368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:04.628237 4368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:04.633567 4375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:04.633579 4377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:04.633602 4374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:04.633812 4368 server_base.cc:1034] running on GCE node
I20260808 20:59:04.634073 4368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:04.634332 4368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:04.635516 4368 hybrid_clock.cc:648] HybridClock initialized: now 1786222744635508 us; error 50 us; skew 500 ppm
Aug 08 20:59:04 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:04.638216 4368 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:59:04.638697 4368 webserver.cc:460] Webserver started at http://127.15.232.131:42569/ using document root <none> and password file <none>
I20260808 20:59:04.638899 4368 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:04.638942 4368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:04.639029 4368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:59:04.640132 4368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "1057c3f7aa044c4ea15c8b023aea66a0"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "dfa8a8bcf346fae81de9d1e7462c518c"
server_key_iv: "9f09244945fde5296f03388388139ed9"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.640452 4368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "1057c3f7aa044c4ea15c8b023aea66a0"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "dfa8a8bcf346fae81de9d1e7462c518c"
server_key_iv: "9f09244945fde5296f03388388139ed9"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.641752 4368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:59:04.642521 4384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.642735 4368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:04.642827 4368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "1057c3f7aa044c4ea15c8b023aea66a0"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "dfa8a8bcf346fae81de9d1e7462c518c"
server_key_iv: "9f09244945fde5296f03388388139ed9"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.642886 4368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:04.657763 4368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:04.659852 4368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:04.659970 4368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:04.660164 4368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:04.660506 4368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:04.660548 4368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.660573 4368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:04.660585 4368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.666731 4368 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:32967
I20260808 20:59:04.666816 4539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:32967 every 8 connection(s)
I20260808 20:59:04.667158 4368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:59:04.673678 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 4368
I20260808 20:59:04.673799 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:59:04.673982 16290 external_mini_cluster.cc:1428] Setting key f5828296d96cd0c237c3fbcd6c067ba6
Aug 08 20:59:04 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:04.677855 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.667931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:49391 (local address 127.15.232.190:38411)
0808 20:59:04.667979 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.667981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.668172 (+ 191us) server_negotiation.cc:407] Connection header received
0808 20:59:04.668763 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.668765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.668779 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.668831 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:04.669838 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.670419 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.671741 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.671943 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.672018 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.674732 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:04.674734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:04.674734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:04.674743 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:04.676202 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.676651 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.676652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.676653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.676801 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.677128 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.677129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.677129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.677262 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:04.677351 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.677661 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.677757 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 20:59:04.678066 4542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.667888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:49391)
0808 20:59:04.668035 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.668084 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:04.668639 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:04.668903 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:04.668907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:04.669529 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:04.669729 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.669734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.670521 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.670522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.671639 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.671642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.672042 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.672043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.672185 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.672777 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:04.672788 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:04.674645 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:04.676302 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:04.676310 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:04.676313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:04.676563 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:04.676906 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:04.676907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:04.676907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:04.677044 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:04.677340 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:04.677348 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:04.677506 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 20:59:04.677627 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 20:59:04.677809 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260808 20:59:04.678573 4540 heartbeater.cc:347] Connected to a master server at 127.15.232.190:38411
I20260808 20:59:04.678691 4540 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:04.678907 4540 heartbeater.cc:511] Master 127.15.232.190:38411 requested a full tablet report, sending...
I20260808 20:59:04.679368 3895 ts_manager.cc:194] Registered new tserver with Master: 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967)
I20260808 20:59:04.679790 3895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:49391
I20260808 20:59:04.686188 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:59:04.694407 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.687773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.15.232.190:38411)
0808 20:59:04.687861 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.687862 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.687970 (+ 108us) server_negotiation.cc:407] Connection header received
0808 20:59:04.688003 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.688004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.688014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.688081 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:04.688660 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.689169 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.689931 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.690149 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.690222 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.691608 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:04.691609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:04.691610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:04.691618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:04.693056 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.693484 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.693485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.693486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.693550 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:04.693797 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:04.693798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:04.693799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:04.693966 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:04.694055 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.694257 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.694335 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 20:59:04.696870 3895 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:59:04.698683 3895 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:59:04.706180 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.701573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:32967 (local address 127.0.0.1:32996)
0808 20:59:04.702184 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.702196 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:04.702252 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:04.702487 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:04.702488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:04.702490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:04.702536 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.702537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.704040 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.704041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.705047 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.705049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.705860 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.705861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.705923 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.705927 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:59:04.705979 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 20:59:04.706065 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":26,"thread_start_us":79,"threads_started":1}
I20260808 20:59:04.706187 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.701573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35965 (local address 127.0.0.1:56088)
0808 20:59:04.701784 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.701848 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:04.701927 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:04.702257 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:04.702261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:04.702264 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:04.702415 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.702417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.703558 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.703559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.704758 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.704761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.705783 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.705783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.705877 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.705892 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 20:59:04.705993 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 20:59:04.706074 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260808 20:59:04.706221 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.701786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56088 (local address 127.15.232.129:35965)
0808 20:59:04.702019 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.702025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.702037 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:59:04.702086 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.702090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.702108 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.702242 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:04.702511 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.703422 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.704865 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.705704 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.705775 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.705838 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.706035 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.706126 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":92,"threads_started":1}
I20260808 20:59:04.706532 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.701895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.15.232.131:32967)
0808 20:59:04.702125 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.702129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.702317 (+ 188us) server_negotiation.cc:407] Connection header received
0808 20:59:04.702365 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.702367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.702385 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.702471 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:04.702647 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.703911 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.705137 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.705679 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.705760 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.705821 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.706348 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.706411 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":117,"threads_started":1}
I20260808 20:59:04.707144 4070 tablet_service.cc:1467] Processing CreateTablet for tablet e06d8ddbe0294d5685e38d272a0c0c15 (DEFAULT_TABLE table=test-table [id=7dee9b6b6aa14044850d98ea5ae44b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:04.707240 4459 tablet_service.cc:1467] Processing CreateTablet for tablet e06d8ddbe0294d5685e38d272a0c0c15 (DEFAULT_TABLE table=test-table [id=7dee9b6b6aa14044850d98ea5ae44b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:04.707576 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.701607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:32785 (local address 127.0.0.1:45234)
0808 20:59:04.702589 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.702600 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:04.702653 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:04.702889 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:04.702890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:04.702892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:04.702934 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.702935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.704327 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.704329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.705585 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.705589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.707261 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.707263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.707340 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.707346 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 20:59:04.707412 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 20:59:04.707490 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
I20260808 20:59:04.707662 4070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06d8ddbe0294d5685e38d272a0c0c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:04.707672 4459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06d8ddbe0294d5685e38d272a0c0c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:04.707918 4559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.701906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45234 (local address 127.15.232.130:32785)
0808 20:59:04.702122 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.702126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.702720 (+ 594us) server_negotiation.cc:407] Connection header received
0808 20:59:04.702768 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.702771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.702787 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.702873 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:04.703008 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.704181 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.705705 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.707157 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.707246 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.707303 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.707782 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.707837 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":105,"threads_started":1}
I20260808 20:59:04.710767 4560 tablet_bootstrap.cc:492] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Bootstrap starting.
I20260808 20:59:04.710793 4561 tablet_bootstrap.cc:492] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: Bootstrap starting.
I20260808 20:59:04.711627 4560 tablet_bootstrap.cc:654] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:04.711632 4561 tablet_bootstrap.cc:654] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:04.711625 4268 tablet_service.cc:1467] Processing CreateTablet for tablet e06d8ddbe0294d5685e38d272a0c0c15 (DEFAULT_TABLE table=test-table [id=7dee9b6b6aa14044850d98ea5ae44b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:04.712029 4268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06d8ddbe0294d5685e38d272a0c0c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:04.712275 4560 log.cc:824] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:04.712311 4561 log.cc:824] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:04.713147 4560 tablet_bootstrap.cc:492] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: No bootstrap required, opened a new log
I20260808 20:59:04.713229 4560 ts_tablet_manager.cc:1397] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 20:59:04.715425 4560 raft_consensus.cc:348] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:04.715679 4560 raft_consensus.cc:374] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:04.716043 4564 tablet_bootstrap.cc:492] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Bootstrap starting.
I20260808 20:59:04.716065 4560 raft_consensus.cc:729] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852838d44a2e4a838b2d5c1c2a3c5db5, State: Initialized, Role: FOLLOWER
I20260808 20:59:04.716274 4560 consensus_queue.cc:260] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:04.716444 4561 tablet_bootstrap.cc:492] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: No bootstrap required, opened a new log
I20260808 20:59:04.716522 4561 ts_tablet_manager.cc:1397] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260808 20:59:04.716639 4560 ts_tablet_manager.cc:1428] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 20:59:04.716847 4167 heartbeater.cc:503] Master 127.15.232.190:38411 was elected leader, sending a full tablet report...
I20260808 20:59:04.717020 4564 tablet_bootstrap.cc:654] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:04.717602 4564 log.cc:824] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:04.718708 4564 tablet_bootstrap.cc:492] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: No bootstrap required, opened a new log
I20260808 20:59:04.718672 4561 raft_consensus.cc:348] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:04.718853 4561 raft_consensus.cc:374] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:04.718900 4561 raft_consensus.cc:729] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1057c3f7aa044c4ea15c8b023aea66a0, State: Initialized, Role: FOLLOWER
I20260808 20:59:04.718829 4564 ts_tablet_manager.cc:1397] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 20:59:04.719043 4561 consensus_queue.cc:260] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:04.719367 4561 ts_tablet_manager.cc:1428] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 20:59:04.719395 4540 heartbeater.cc:503] Master 127.15.232.190:38411 was elected leader, sending a full tablet report...
I20260808 20:59:04.721318 4564 raft_consensus.cc:348] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:04.721505 4564 raft_consensus.cc:374] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:04.721557 4564 raft_consensus.cc:729] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95181f5851114da09ff09791d505abd5, State: Initialized, Role: FOLLOWER
I20260808 20:59:04.721772 4564 consensus_queue.cc:260] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:04.722163 4564 ts_tablet_manager.cc:1428] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 20:59:04.722201 4359 heartbeater.cc:503] Master 127.15.232.190:38411 was elected leader, sending a full tablet report...
W20260808 20:59:04.754885 4361 tablet.cc:2369] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:59:04.792932 4567 raft_consensus.cc:482] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:59:04.793147 4567 raft_consensus.cc:504] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:04.793625 4567 leader_election.cc:288] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967), 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785)
I20260808 20:59:04.797335 4559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.794010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:52443 (local address 127.15.232.130:32785)
0808 20:59:04.794082 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.794084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.794096 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:59:04.794128 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.794129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.794141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.794198 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:04.794380 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.794937 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.796454 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.796852 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.796928 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.796936 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.797193 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.797239 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 20:59:04.797389 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.794128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:45535 (local address 127.15.232.131:32967)
0808 20:59:04.794197 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.794200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.794224 (+ 24us) server_negotiation.cc:407] Connection header received
0808 20:59:04.794249 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.794250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.794259 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.794314 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:04.794570 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.795159 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.796281 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.796861 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.797001 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.797015 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.797232 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.797286 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 20:59:04.797398 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.793883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:32785 (local address 127.15.232.129:52443)
0808 20:59:04.793952 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.793964 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:04.794009 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:04.794216 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:04.794218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:04.794220 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:04.794300 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.794302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.795043 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.795045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.796255 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.796266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.796971 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.796973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.797092 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.797097 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:59:04.797163 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 20:59:04.797267 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":33}
I20260808 20:59:04.797626 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.793904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:32967 (local address 127.15.232.129:45535)
0808 20:59:04.794085 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.794097 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:04.794172 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:04.794328 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:04.794328 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:04.794330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:04.794430 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.794431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.795259 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.795260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.796080 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:04.796083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.797037 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.797038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:04.797110 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.797115 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:59:04.797201 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 20:59:04.797253 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260808 20:59:04.797955 4297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06d8ddbe0294d5685e38d272a0c0c15" candidate_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95181f5851114da09ff09791d505abd5" is_pre_election: true
I20260808 20:59:04.798169 4297 raft_consensus.cc:2454] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 852838d44a2e4a838b2d5c1c2a3c5db5 in term 0.
I20260808 20:59:04.798225 4481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06d8ddbe0294d5685e38d272a0c0c15" candidate_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" is_pre_election: true
I20260808 20:59:04.798456 4481 raft_consensus.cc:2454] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 852838d44a2e4a838b2d5c1c2a3c5db5 in term 0.
I20260808 20:59:04.798487 4005 leader_election.cc:302] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 852838d44a2e4a838b2d5c1c2a3c5db5, 95181f5851114da09ff09791d505abd5; no voters:
I20260808 20:59:04.798686 4567 raft_consensus.cc:2781] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:59:04.798756 4567 raft_consensus.cc:482] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:59:04.798796 4567 raft_consensus.cc:3037] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:04.799762 4567 raft_consensus.cc:504] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:04.799950 4567 leader_election.cc:288] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 1 election: Requested vote from peers 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967), 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785)
I20260808 20:59:04.800134 4297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06d8ddbe0294d5685e38d272a0c0c15" candidate_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95181f5851114da09ff09791d505abd5"
I20260808 20:59:04.800287 4297 raft_consensus.cc:3037] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:04.800232 4481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06d8ddbe0294d5685e38d272a0c0c15" candidate_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1057c3f7aa044c4ea15c8b023aea66a0"
I20260808 20:59:04.800385 4481 raft_consensus.cc:3037] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:04.801164 4481 raft_consensus.cc:2454] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 852838d44a2e4a838b2d5c1c2a3c5db5 in term 1.
I20260808 20:59:04.801250 4297 raft_consensus.cc:2454] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 852838d44a2e4a838b2d5c1c2a3c5db5 in term 1.
I20260808 20:59:04.801375 4008 leader_election.cc:302] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1057c3f7aa044c4ea15c8b023aea66a0, 852838d44a2e4a838b2d5c1c2a3c5db5; no voters:
I20260808 20:59:04.801491 4567 raft_consensus.cc:2781] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:59:04.801678 4567 raft_consensus.cc:686] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 LEADER]: Becoming Leader. State: Replica: 852838d44a2e4a838b2d5c1c2a3c5db5, State: Running, Role: LEADER
I20260808 20:59:04.801797 4567 consensus_queue.cc:237] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:04.802845 3896 catalog_manager.cc:5495] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 852838d44a2e4a838b2d5c1c2a3c5db5 (127.15.232.129). New cstate: current_term: 1 leader_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } health_report { overall_health: UNKNOWN } } }
W20260808 20:59:04.824663 4168 tablet.cc:2369] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:59:04.828047 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.825020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56090 (local address 127.15.232.129:35965)
0808 20:59:04.825080 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:04.825083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:04.825095 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:59:04.825142 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:04.825142 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:04.825161 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:04.825240 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:59:04.825387 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.825910 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.826814 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:04.827028 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:04.827112 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:04.827356 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:59:04.827657 (+ 301us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:59:04.827881 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:04.827937 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 20:59:04.827960 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 20:59:04.830179 4481 raft_consensus.cc:1261] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [term 1 FOLLOWER]: Refusing update from remote peer 852838d44a2e4a838b2d5c1c2a3c5db5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:59:04.830222 4297 raft_consensus.cc:1261] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [term 1 FOLLOWER]: Refusing update from remote peer 852838d44a2e4a838b2d5c1c2a3c5db5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:59:04.830797 4575 consensus_queue.cc:1035] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:59:04.831070 4567 consensus_queue.cc:1035] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:59:04.834393 4580 mvcc.cc:205] Tried to move back new op lower bound from 7316368362821246976 to 7316368362709090304. Current Snapshot: MvccSnapshot[applied={T|T < 7316368362821246976}]
I20260808 20:59:04.835757 3896 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:59:04.835872 3896 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:59:04.836742 3896 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Sending DeleteTablet for 3 replicas of tablet e06d8ddbe0294d5685e38d272a0c0c15
I20260808 20:59:04.837083 4459 tablet_service.cc:1514] Processing DeleteTablet for tablet e06d8ddbe0294d5685e38d272a0c0c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:04 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:32996
I20260808 20:59:04.837152 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 3977
I20260808 20:59:04.837149 4268 tablet_service.cc:1514] Processing DeleteTablet for tablet e06d8ddbe0294d5685e38d272a0c0c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:04 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:45234
I20260808 20:59:04.837442 4591 tablet_replica.cc:330] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: stopping tablet replica
I20260808 20:59:04.837577 4591 raft_consensus.cc:2229] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:59:04.837714 4591 raft_consensus.cc:2258] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:04.841289 4591 ts_tablet_manager.cc:1905] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:04.843250 4591 ts_tablet_manager.cc:1918] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:04.843423 4591 log.cc:1196] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/e06d8ddbe0294d5685e38d272a0c0c15
I20260808 20:59:04.843786 4591 ts_tablet_manager.cc:1939] T e06d8ddbe0294d5685e38d272a0c0c15 P 1057c3f7aa044c4ea15c8b023aea66a0: Deleting consensus metadata
I20260808 20:59:04.844355 3877 catalog_manager.cc:4841] TS 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967): tablet e06d8ddbe0294d5685e38d272a0c0c15 (table test-table [id=7dee9b6b6aa14044850d98ea5ae44b22]) successfully deleted
I20260808 20:59:04.851717 4594 tablet_replica.cc:330] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: stopping tablet replica
I20260808 20:59:04.851924 4594 raft_consensus.cc:2229] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:59:04.852088 4594 raft_consensus.cc:2258] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:04.852662 4594 ts_tablet_manager.cc:1905] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:04.854382 4594 ts_tablet_manager.cc:1918] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:04.854717 4594 log.cc:1196] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/e06d8ddbe0294d5685e38d272a0c0c15
W20260808 20:59:04.854735 3876 connection.cc:535] client connection to 127.15.232.129:35965 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 20:59:04.854856 3876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 20:59:04.855083 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 4173
I20260808 20:59:04.855183 4594 ts_tablet_manager.cc:1939] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Deleting consensus metadata
I20260808 20:59:04.855908 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.855776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35965 (local address 127.0.0.1:56102)
0808 20:59:04.855831 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.855867 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:35965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 20:59:04.856099 3876 catalog_manager.cc:4584] TS 852838d44a2e4a838b2d5c1c2a3c5db5 (127.15.232.129:35965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e06d8ddbe0294d5685e38d272a0c0c15: Network error: Client connection negotiation failed: client connection to 127.15.232.129:35965: connect: Connection refused (error 111)
I20260808 20:59:04.867293 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 4368
I20260808 20:59:04.867372 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:04.867203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:32785 (local address 127.0.0.1:45242)
0808 20:59:04.867298 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:04.867333 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.130:32785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 20:59:04.876515 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 3852
I20260808 20:59:04.888288 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:38411
--webserver_interface=127.15.232.190
--webserver_port=33249
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:38411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:04.974502 4597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:04.974709 4597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:04.974731 4597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:04.975932 4597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:59:04.975977 4597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:04.975992 4597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:04.976004 4597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:59:04.976016 4597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:59:04.977551 4597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:38411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:38411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:04.977810 4597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:04.978049 4597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:04.987928 4608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:04.987942 4609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:04.988008 4612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:04.988410 4597 server_base.cc:1034] running on GCE node
I20260808 20:59:04.988718 4597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:04.989060 4597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:04.990232 4597 hybrid_clock.cc:648] HybridClock initialized: now 1786222744990220 us; error 32 us; skew 500 ppm
Aug 08 20:59:04 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:04.993053 4597 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:59:04.993588 4597 webserver.cc:460] Webserver started at http://127.15.232.190:33249/ using document root <none> and password file <none>
I20260808 20:59:04.993822 4597 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:04.993939 4597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:04.995396 4597 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 20:59:04.996228 4620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:04.996465 4597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:04.996587 4597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "db57001c48ac4b67891f56b4db0ca97a"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "20e8ddc02769ae6a25834668f2ee90a1"
server_key_iv: "cd093e8822e8a3944a68d854fb50993c"
server_key_version: "encryptionkey@0"
I20260808 20:59:04.997041 4597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:05.005820 4597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:05.008529 4597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:05.008704 4597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:05.013971 4597 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:38411
I20260808 20:59:05.014015 4691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:38411 every 8 connection(s)
I20260808 20:59:05.014525 4597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:59:05.014802 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 4597
I20260808 20:59:05.015195 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:35965
--local_ip_for_outbound_sockets=127.15.232.129
--tserver_master_addrs=127.15.232.190:38411
--webserver_port=36759
--webserver_interface=127.15.232.129
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:59:05.016707 4693 sys_catalog.cc:263] Verifying existing consensus state
I20260808 20:59:05.017895 4693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Bootstrap starting.
I20260808 20:59:05.027770 4693 log.cc:824] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:05.030133 4693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:59:05.030505 4693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Bootstrap complete.
I20260808 20:59:05.033209 4693 raft_consensus.cc:348] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:05.033461 4693 raft_consensus.cc:729] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db57001c48ac4b67891f56b4db0ca97a, State: Initialized, Role: FOLLOWER
I20260808 20:59:05.033664 4693 consensus_queue.cc:260] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:05.033872 4693 raft_consensus.cc:388] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:59:05.033972 4693 raft_consensus.cc:482] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:59:05.034025 4693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 1 FOLLOWER]: Advancing to term 2
I20260808 20:59:05.035323 4693 raft_consensus.cc:504] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:05.035418 4693 leader_election.cc:302] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db57001c48ac4b67891f56b4db0ca97a; no voters:
I20260808 20:59:05.035734 4693 leader_election.cc:288] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 20:59:05.035784 4698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 20:59:05.035965 4698 raft_consensus.cc:686] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 2 LEADER]: Becoming Leader. State: Replica: db57001c48ac4b67891f56b4db0ca97a, State: Running, Role: LEADER
I20260808 20:59:05.036032 4693 sys_catalog.cc:564] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:59:05.036134 4698 consensus_queue.cc:237] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:05.036435 4698 sys_catalog.cc:455] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: SysCatalogTable state changed. Reason: New leader db57001c48ac4b67891f56b4db0ca97a. Latest consensus state: current_term: 2 leader_uuid: "db57001c48ac4b67891f56b4db0ca97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } } }
I20260808 20:59:05.036438 4702 sys_catalog.cc:455] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "db57001c48ac4b67891f56b4db0ca97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } } }
I20260808 20:59:05.036643 4698 sys_catalog.cc:458] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:05.036741 4702 sys_catalog.cc:458] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:05.036906 4710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:59:05.037884 4710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:59:05.038074 4710 catalog_manager.cc:1259] Loaded cluster ID: fa129c8b739d4574b543303e9980170b
I20260808 20:59:05.038336 4710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:59:05.038667 4710 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 20:59:05.043356 4710 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 20:59:05.046268 4710 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:59:05.046550 4710 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Loaded TSK: 0
I20260808 20:59:05.046798 4710 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 20:59:05.115877 4695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:05.116117 4695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:05.116149 4695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:05.117548 4695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:05.117611 4695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:05.117647 4695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:59:05.119354 4695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:35965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=36759
--tserver_master_addrs=127.15.232.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:05.119707 4695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:05.120002 4695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:05.125995 4725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.125962 4726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.125988 4728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:05.126497 4695 server_base.cc:1034] running on GCE node
I20260808 20:59:05.126703 4695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:05.127003 4695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:05.128254 4695 hybrid_clock.cc:648] HybridClock initialized: now 1786222745128232 us; error 32 us; skew 500 ppm
Aug 08 20:59:05 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:05.131352 4695 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:59:05.131902 4695 webserver.cc:460] Webserver started at http://127.15.232.129:36759/ using document root <none> and password file <none>
I20260808 20:59:05.132146 4695 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:05.132236 4695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:05.133728 4695 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:59:05.134473 4739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:05.134719 4695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:05.134850 4695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "852838d44a2e4a838b2d5c1c2a3c5db5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "c449d90c5362de319eb05d4a9a232bc9"
server_key_iv: "6151816ae314024abb60ba001379ad19"
server_key_version: "encryptionkey@0"
I20260808 20:59:05.135322 4695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:05.165637 4695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:05.168702 4695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:05.168866 4695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:05.169142 4695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:05.169710 4749 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 20:59:05.171312 4695 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 20:59:05.171393 4695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 20:59:05.171468 4695 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 20:59:05.172401 4695 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 20:59:05.172459 4695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 20:59:05.172652 4749 tablet_bootstrap.cc:492] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Bootstrap starting.
I20260808 20:59:05.179904 4695 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:35965
I20260808 20:59:05.180154 4902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:35965 every 8 connection(s)
I20260808 20:59:05.180450 4695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:59:05.182904 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 4695
I20260808 20:59:05.183281 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:32785
--local_ip_for_outbound_sockets=127.15.232.130
--tserver_master_addrs=127.15.232.190:38411
--webserver_port=42449
--webserver_interface=127.15.232.130
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:59:05.184585 4749 log.cc:824] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:05.187845 4749 tablet_bootstrap.cc:492] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:59:05.188201 4749 tablet_bootstrap.cc:492] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Bootstrap complete.
I20260808 20:59:05.189384 4749 ts_tablet_manager.cc:1397] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.012s
I20260808 20:59:05.191210 4749 raft_consensus.cc:348] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
I20260808 20:59:05.191681 4749 raft_consensus.cc:729] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852838d44a2e4a838b2d5c1c2a3c5db5, State: Initialized, Role: FOLLOWER
I20260808 20:59:05.191861 4749 consensus_queue.cc:260] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } } peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } }
Aug 08 20:59:05 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:05.192642 4749 ts_tablet_manager.cc:1428] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 20:59:05.198172 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.181219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:56173)
0808 20:59:05.181444 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.181494 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:05.182163 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:05.184590 (+ 2427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:05.184595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:05.185027 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:05.185346 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.185355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.187308 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.187310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.188725 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.188729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.189686 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.189688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.189833 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.190545 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:05.190556 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:05.193083 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:05.195960 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:05.195969 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:05.195972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:05.196249 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:05.196598 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:05.196599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:05.196600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:05.196891 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:05.197503 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:05.197512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:05.197688 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 20:59:05.197811 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 20:59:05.197992 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
I20260808 20:59:05.198330 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.181322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:56173 (local address 127.15.232.190:38411)
0808 20:59:05.183203 (+ 1881us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.183215 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.183265 (+ 50us) server_negotiation.cc:407] Connection header received
0808 20:59:05.183468 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.183477 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.183565 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.183756 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:05.185524 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.187153 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.188834 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.189550 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.189704 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.193230 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:05.193236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:05.193236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:05.193252 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:05.195833 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.196349 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.196354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.196355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.196513 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.196985 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.196986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.196987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.197372 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:05.197474 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.197934 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.198128 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1819,"thread_start_us":92,"threads_started":1}
I20260808 20:59:05.199203 4903 heartbeater.cc:347] Connected to a master server at 127.15.232.190:38411
I20260808 20:59:05.199354 4903 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:05.199635 4903 heartbeater.cc:511] Master 127.15.232.190:38411 requested a full tablet report, sending...
I20260808 20:59:05.200688 4639 ts_manager.cc:194] Registered new tserver with Master: 852838d44a2e4a838b2d5c1c2a3c5db5 (127.15.232.129:35965)
I20260808 20:59:05.202134 4639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:56173
I20260808 20:59:05.203172 4903 heartbeater.cc:503] Master 127.15.232.190:38411 was elected leader, sending a full tablet report...
Aug 08 20:59:05 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:59:05.211552 4914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.201955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56118 (local address 127.15.232.129:35965)
0808 20:59:05.202142 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.202146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.202153 (+ 7us) server_negotiation.cc:407] Connection header received
0808 20:59:05.202183 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.202184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.202196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.202282 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:05.202982 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.204126 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.204943 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.205419 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.205497 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.207883 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:05.207892 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:05.207892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:05.207900 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:05.210097 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.210547 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.210551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.210551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.210623 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.210863 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.210864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.210865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.211057 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:05.211161 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.211321 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.211457 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260808 20:59:05.211565 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.201528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35965 (local address 127.0.0.1:56118)
0808 20:59:05.201825 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.201881 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:05.201971 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:05.202351 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:05.202395 (+ 44us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:05.202818 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:05.202885 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.202888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.204230 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.204231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.204848 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.204851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.205524 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.205525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.205597 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.206170 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:05.206176 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:05.207795 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:05.210197 (+ 2402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:05.210204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:05.210206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:05.210460 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:05.210715 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:05.210716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:05.210716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:05.210788 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:05.211139 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:05.211146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:05.211195 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 20:59:05.211264 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 20:59:05.211380 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":205,"threads_started":1}
I20260808 20:59:05.212037 4814 tablet_service.cc:1514] Processing DeleteTablet for tablet e06d8ddbe0294d5685e38d272a0c0c15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:04 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:56118
I20260808 20:59:05.212677 4915 tablet_replica.cc:330] stopping tablet replica
I20260808 20:59:05.212795 4915 raft_consensus.cc:2229] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:59:05.212886 4915 raft_consensus.cc:2258] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:05.213227 4915 ts_tablet_manager.cc:1905] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:05.215147 4915 ts_tablet_manager.cc:1918] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:05.215215 4915 log.cc:1196] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/e06d8ddbe0294d5685e38d272a0c0c15
I20260808 20:59:05.215502 4915 ts_tablet_manager.cc:1939] T e06d8ddbe0294d5685e38d272a0c0c15 P 852838d44a2e4a838b2d5c1c2a3c5db5: Deleting consensus metadata
I20260808 20:59:05.216019 4624 catalog_manager.cc:4841] TS 852838d44a2e4a838b2d5c1c2a3c5db5 (127.15.232.129:35965): tablet e06d8ddbe0294d5685e38d272a0c0c15 (table test-table [id=7dee9b6b6aa14044850d98ea5ae44b22]) successfully deleted
W20260808 20:59:05.280474 4909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:05.280763 4909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:05.280810 4909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:05.282114 4909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:05.282192 4909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:05.282240 4909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:59:05.283824 4909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:32785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=42449
--tserver_master_addrs=127.15.232.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:05.284137 4909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:05.284394 4909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:05.290746 4924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.290825 4923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.290802 4927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:05.291023 4909 server_base.cc:1034] running on GCE node
I20260808 20:59:05.291283 4909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:05.291589 4909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:05.292788 4909 hybrid_clock.cc:648] HybridClock initialized: now 1786222745292775 us; error 38 us; skew 500 ppm
Aug 08 20:59:05 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:05.296239 4909 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:59:05.296810 4909 webserver.cc:460] Webserver started at http://127.15.232.130:42449/ using document root <none> and password file <none>
I20260808 20:59:05.297070 4909 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:05.297134 4909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:05.298808 4909 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:05.299718 4938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:05.299959 4909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:05.300074 4909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "95181f5851114da09ff09791d505abd5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "148f2775c79a49cdcb288f7cd5a273c5"
server_key_iv: "27a322db7a7d82aaf38c16629adca2c1"
server_key_version: "encryptionkey@0"
I20260808 20:59:05.300459 4909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:05.313362 4909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:05.315407 4909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:05.315616 4909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:05.315837 4909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:05.316254 4956 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260808 20:59:05.317734 4956 ts_tablet_manager.cc:1852] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 20:59:05.318027 4956 ts_tablet_manager.cc:1905] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:05.319998 4956 ts_tablet_manager.cc:1918] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:05.320147 4956 ts_tablet_manager.cc:1939] T e06d8ddbe0294d5685e38d272a0c0c15 P 95181f5851114da09ff09791d505abd5: Deleting consensus metadata
I20260808 20:59:05.320533 4909 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 20:59:05.320619 4909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 20:59:05.320691 4909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:05.320724 4909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:05.327337 4909 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:32785
I20260808 20:59:05.327387 5108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:32785 every 8 connection(s)
I20260808 20:59:05.327759 4909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:59:05.331933 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 4909
I20260808 20:59:05.332302 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:32967
--local_ip_for_outbound_sockets=127.15.232.131
--tserver_master_addrs=127.15.232.190:38411
--webserver_port=42569
--webserver_interface=127.15.232.131
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 20:59:05 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:05.338362 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.328550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:60283 (local address 127.15.232.190:38411)
0808 20:59:05.328622 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.328624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.328781 (+ 157us) server_negotiation.cc:407] Connection header received
0808 20:59:05.329326 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.329327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.329336 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.329399 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:05.330056 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.330645 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.331811 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.332034 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.332127 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.334908 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:05.334910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:05.334910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:05.334921 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:05.336756 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.337307 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.337309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.337309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.337382 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.337648 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.337650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.337650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.337786 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:05.337868 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.338162 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.338265 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 20:59:05.338546 5112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.328485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:60283)
0808 20:59:05.328652 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.328697 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:05.329233 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:05.329497 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:05.329500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:05.329762 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:05.329967 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.329972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.330763 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.330764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.331683 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.331687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.332148 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.332149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.332302 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.332900 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:05.332921 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:05.334789 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:05.336857 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:05.336864 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:05.336866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:05.337172 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:05.337454 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:05.337454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:05.337455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:05.337560 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:05.337858 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:05.337866 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:05.338005 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 20:59:05.338127 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 20:59:05.338296 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260808 20:59:05.339221 5109 heartbeater.cc:347] Connected to a master server at 127.15.232.190:38411
I20260808 20:59:05.339341 5109 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:05.339584 5109 heartbeater.cc:511] Master 127.15.232.190:38411 requested a full tablet report, sending...
I20260808 20:59:05.340039 4639 ts_manager.cc:194] Registered new tserver with Master: 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785)
I20260808 20:59:05.340643 4639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:60283
W20260808 20:59:05.420055 5114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:05.420266 5114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:05.420291 5114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:05.421397 5114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:05.421433 5114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:05.421460 5114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:59:05.422888 5114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:32967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=42569
--tserver_master_addrs=127.15.232.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:05.423105 5114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:05.423293 5114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:05.428534 5127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.428516 5125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.428555 5120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:05.428964 5114 server_base.cc:1034] running on GCE node
I20260808 20:59:05.429231 5114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:05.429519 5114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:05.430673 5114 hybrid_clock.cc:648] HybridClock initialized: now 1786222745430676 us; error 47 us; skew 500 ppm
Aug 08 20:59:05 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:05.433368 5114 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:59:05.433857 5114 webserver.cc:460] Webserver started at http://127.15.232.131:42569/ using document root <none> and password file <none>
I20260808 20:59:05.434123 5114 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:05.434212 5114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:05.435775 5114 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:05.436664 5139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:05.436978 5114 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:05.437099 5114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "1057c3f7aa044c4ea15c8b023aea66a0"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "dfa8a8bcf346fae81de9d1e7462c518c"
server_key_iv: "9f09244945fde5296f03388388139ed9"
server_key_version: "encryptionkey@0"
I20260808 20:59:05.437556 5114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:05.457430 5114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:05.459969 5114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:05.460158 5114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:05.460455 5114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:05.460804 5114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:05.460862 5114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:05.460901 5114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:05.460947 5114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:05.467051 5114 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:32967
I20260808 20:59:05.467097 5306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:32967 every 8 connection(s)
I20260808 20:59:05.467535 5114 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:59:05.469959 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 5114
Aug 08 20:59:05 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:05.477886 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.470426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.15.232.190:38411)
0808 20:59:05.470596 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.470598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.470649 (+ 51us) server_negotiation.cc:407] Connection header received
0808 20:59:05.470743 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.470744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.470756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.470849 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:05.471630 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.472183 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.473116 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.473416 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.473495 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.474909 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:05.474910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:05.474911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:05.474919 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:05.476634 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.477045 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.477048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.477048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.477108 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.477320 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.477321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.477321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.477447 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:05.477634 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.477676 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.477769 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":72,"threads_started":1}
I20260808 20:59:05.478332 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.468397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:57451 (local address 127.15.232.190:38411)
0808 20:59:05.468494 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.468496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.468733 (+ 237us) server_negotiation.cc:407] Connection header received
0808 20:59:05.469243 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.469244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.469256 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.469321 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:05.470556 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.471077 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.472174 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.472374 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.472460 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.474828 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:05.474830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:05.474830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:05.474839 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:05.476749 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.477208 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.477210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.477210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.477272 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:05.477531 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:05.477532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:05.477532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:05.477652 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:05.477749 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.478163 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.478242 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 20:59:05.478503 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.468347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:57451)
0808 20:59:05.468516 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.468559 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:05.469111 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:05.469403 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:05.469406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:05.470193 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:05.470465 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.470474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.471186 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.471187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.472075 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.472078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.472475 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.472476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.472605 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.473234 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:05.473244 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:05.474728 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:05.476879 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:05.476887 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:05.476889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:05.477128 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:05.477351 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:05.477351 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:05.477352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:05.477463 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:05.477745 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:05.477753 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:05.477939 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 20:59:05.478057 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 20:59:05.478255 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260808 20:59:05.478965 5307 heartbeater.cc:347] Connected to a master server at 127.15.232.190:38411
I20260808 20:59:05.479080 5307 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:05.479274 5307 heartbeater.cc:511] Master 127.15.232.190:38411 requested a full tablet report, sending...
I20260808 20:59:05.479820 4639 ts_manager.cc:194] Registered new tserver with Master: 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967)
I20260808 20:59:05.480257 4639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:57451
I20260808 20:59:05.489813 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:59:05.493047 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.490357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.15.232.190:38411)
0808 20:59:05.490419 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.490421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.490620 (+ 199us) server_negotiation.cc:407] Connection header received
0808 20:59:05.490665 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.490672 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.490717 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.490772 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:59:05.490910 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.491401 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.492264 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.492454 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.492508 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.492628 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:59:05.492740 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:59:05.492801 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.492929 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.492947 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 20:59:05.493418 4639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:59:05.495095 4639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:59:05.497629 4814 tablet_service.cc:1467] Processing CreateTablet for tablet 81b81b5e8c314adebb8048bbc45742ba (DEFAULT_TABLE table=test-table [id=8846b7aace1045419a76ac0efd108cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:05.497859 4814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81b81b5e8c314adebb8048bbc45742ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:05.499306 4749 tablet_bootstrap.cc:492] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Bootstrap starting.
I20260808 20:59:05.499738 4749 tablet_bootstrap.cc:654] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:05.500335 4749 tablet_bootstrap.cc:492] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: No bootstrap required, opened a new log
I20260808 20:59:05.500402 4749 ts_tablet_manager.cc:1397] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 20:59:05.500767 4749 raft_consensus.cc:348] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.500859 4749 raft_consensus.cc:374] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:05.500903 4749 raft_consensus.cc:729] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852838d44a2e4a838b2d5c1c2a3c5db5, State: Initialized, Role: FOLLOWER
I20260808 20:59:05.500980 4749 consensus_queue.cc:260] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.501084 4749 ts_tablet_manager.cc:1428] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 20:59:05.501906 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.497373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:32967 (local address 127.0.0.1:33000)
0808 20:59:05.497503 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.497540 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:05.497580 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:05.497950 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:05.497951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:05.497952 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:05.497984 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.497985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.499320 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.499321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.500661 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.500664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.501627 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.501628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.501698 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.501724 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 20:59:05.501757 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 20:59:05.501803 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":8,"thread_start_us":66,"threads_started":1}
I20260808 20:59:05.502208 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.497303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:32785 (local address 127.0.0.1:45246)
0808 20:59:05.497355 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.497366 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:05.497405 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:05.497907 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:05.497908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:05.497910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:05.497963 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.497965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.499452 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.499453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.500700 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.500704 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.501941 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.501942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.502032 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.502037 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:59:05.502092 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 20:59:05.502137 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":35}
I20260808 20:59:05.502230 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.497504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.15.232.130:32785)
0808 20:59:05.497713 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.497718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.497729 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:59:05.497775 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.497777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.497793 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.497884 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:05.498078 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.499326 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.500793 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.501838 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.501942 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.502011 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.502107 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.502148 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":129,"threads_started":1}
I20260808 20:59:05.502314 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.497559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33000 (local address 127.15.232.131:32967)
0808 20:59:05.497753 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.497757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.497770 (+ 13us) server_negotiation.cc:407] Connection header received
0808 20:59:05.497811 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.497812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.497825 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.498061 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:05.498143 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.499202 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.500767 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.501517 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.502089 (+ 572us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.502150 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.502176 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.502217 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260808 20:59:05.502811 5013 tablet_service.cc:1467] Processing CreateTablet for tablet 81b81b5e8c314adebb8048bbc45742ba (DEFAULT_TABLE table=test-table [id=8846b7aace1045419a76ac0efd108cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:05.502934 5215 tablet_service.cc:1467] Processing CreateTablet for tablet 81b81b5e8c314adebb8048bbc45742ba (DEFAULT_TABLE table=test-table [id=8846b7aace1045419a76ac0efd108cdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:05.503165 5013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81b81b5e8c314adebb8048bbc45742ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:05.503294 5215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81b81b5e8c314adebb8048bbc45742ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:05.505115 4956 tablet_bootstrap.cc:492] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: Bootstrap starting.
I20260808 20:59:05.505785 4956 tablet_bootstrap.cc:654] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:05.506383 4956 log.cc:824] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:05.507005 5326 tablet_bootstrap.cc:492] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: Bootstrap starting.
I20260808 20:59:05.507225 4956 tablet_bootstrap.cc:492] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: No bootstrap required, opened a new log
I20260808 20:59:05.507289 4956 ts_tablet_manager.cc:1397] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 20:59:05.507764 5326 tablet_bootstrap.cc:654] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:05.508266 5326 log.cc:824] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:05.509109 5326 tablet_bootstrap.cc:492] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: No bootstrap required, opened a new log
I20260808 20:59:05.509213 5326 ts_tablet_manager.cc:1397] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:59:05.509151 4956 raft_consensus.cc:348] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.509337 4956 raft_consensus.cc:374] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:05.509392 4956 raft_consensus.cc:729] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95181f5851114da09ff09791d505abd5, State: Initialized, Role: FOLLOWER
I20260808 20:59:05.509510 4956 consensus_queue.cc:260] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.509824 5109 heartbeater.cc:503] Master 127.15.232.190:38411 was elected leader, sending a full tablet report...
I20260808 20:59:05.509845 4956 ts_tablet_manager.cc:1428] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 20:59:05.511264 5326 raft_consensus.cc:348] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.511416 5326 raft_consensus.cc:374] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:05.511502 5326 raft_consensus.cc:729] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1057c3f7aa044c4ea15c8b023aea66a0, State: Initialized, Role: FOLLOWER
I20260808 20:59:05.511629 5326 consensus_queue.cc:260] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.511934 5307 heartbeater.cc:503] Master 127.15.232.190:38411 was elected leader, sending a full tablet report...
I20260808 20:59:05.511941 5326 ts_tablet_manager.cc:1428] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 20:59:05.555604 4911 raft_consensus.cc:482] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:59:05.555799 4911 raft_consensus.cc:504] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.556207 4911 leader_election.cc:288] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785), 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967)
I20260808 20:59:05.559957 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.556523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:49283 (local address 127.15.232.131:32967)
0808 20:59:05.556594 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.556596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.556654 (+ 58us) server_negotiation.cc:407] Connection header received
0808 20:59:05.556686 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.556687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.556697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.557027 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:05.557081 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.557852 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.559018 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.559558 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.559618 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.559627 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.559833 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.559865 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 20:59:05.559978 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.556425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:32967 (local address 127.15.232.129:49283)
0808 20:59:05.556489 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.556503 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:05.556572 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:05.556864 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:05.556865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:05.556867 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:05.556949 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.556951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.558037 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.558038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.558911 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.558914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.559635 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.559637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.559725 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.559753 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 20:59:05.559820 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 20:59:05.559873 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":5}
I20260808 20:59:05.560248 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.556692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:57159 (local address 127.15.232.130:32785)
0808 20:59:05.556752 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.556754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.557228 (+ 474us) server_negotiation.cc:407] Connection header received
0808 20:59:05.557265 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.557266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.557280 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.557342 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:05.557519 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.558163 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.559239 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.559846 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.559913 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.559921 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.560139 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.560184 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 20:59:05.560245 5334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.556451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:32785 (local address 127.15.232.129:57159)
0808 20:59:05.557131 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.557143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:05.557218 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:05.557345 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:05.557345 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:05.557347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:05.557440 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.557441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.558225 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.558226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.559144 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:05.559148 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.559928 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.559929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:05.560005 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.560012 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 20:59:05.560124 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 20:59:05.560175 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":98,"threads_started":1}
I20260808 20:59:05.560606 5042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81b81b5e8c314adebb8048bbc45742ba" candidate_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95181f5851114da09ff09791d505abd5" is_pre_election: true
I20260808 20:59:05.560639 5246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81b81b5e8c314adebb8048bbc45742ba" candidate_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" is_pre_election: true
I20260808 20:59:05.560770 5042 raft_consensus.cc:2454] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 852838d44a2e4a838b2d5c1c2a3c5db5 in term 0.
I20260808 20:59:05.560838 5246 raft_consensus.cc:2454] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 852838d44a2e4a838b2d5c1c2a3c5db5 in term 0.
I20260808 20:59:05.561034 4741 leader_election.cc:302] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 852838d44a2e4a838b2d5c1c2a3c5db5, 95181f5851114da09ff09791d505abd5; no voters:
I20260808 20:59:05.561163 4911 raft_consensus.cc:2781] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:59:05.561221 4911 raft_consensus.cc:482] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:59:05.561244 4911 raft_consensus.cc:3037] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:05.561942 4911 raft_consensus.cc:504] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.562100 4911 leader_election.cc:288] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 1 election: Requested vote from peers 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785), 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967)
I20260808 20:59:05.562244 5042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81b81b5e8c314adebb8048bbc45742ba" candidate_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95181f5851114da09ff09791d505abd5"
I20260808 20:59:05.562278 5246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81b81b5e8c314adebb8048bbc45742ba" candidate_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1057c3f7aa044c4ea15c8b023aea66a0"
I20260808 20:59:05.562361 5246 raft_consensus.cc:3037] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:05.562366 5042 raft_consensus.cc:3037] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:05.563102 5246 raft_consensus.cc:2454] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 852838d44a2e4a838b2d5c1c2a3c5db5 in term 1.
I20260808 20:59:05.563221 5042 raft_consensus.cc:2454] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 852838d44a2e4a838b2d5c1c2a3c5db5 in term 1.
I20260808 20:59:05.563298 4744 leader_election.cc:302] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1057c3f7aa044c4ea15c8b023aea66a0, 852838d44a2e4a838b2d5c1c2a3c5db5; no voters:
I20260808 20:59:05.563395 4911 raft_consensus.cc:2781] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:59:05.563588 4911 raft_consensus.cc:686] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 LEADER]: Becoming Leader. State: Replica: 852838d44a2e4a838b2d5c1c2a3c5db5, State: Running, Role: LEADER
I20260808 20:59:05.563691 4911 consensus_queue.cc:237] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.564566 4639 catalog_manager.cc:5495] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 852838d44a2e4a838b2d5c1c2a3c5db5 (127.15.232.129). New cstate: current_term: 1 leader_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } health_report { overall_health: HEALTHY } } }
I20260808 20:59:05.577896 4914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.575498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56122 (local address 127.15.232.129:35965)
0808 20:59:05.575570 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:05.575572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:05.575602 (+ 30us) server_negotiation.cc:407] Connection header received
0808 20:59:05.575687 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:05.575688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:05.575707 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:05.575771 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:59:05.575898 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.576385 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.577107 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:05.577330 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:05.577387 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:05.577550 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:59:05.577657 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:59:05.577713 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:05.577767 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 20:59:05.577787 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 20:59:05.578505 5111 tablet.cc:2369] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:59:05.579680 5042 raft_consensus.cc:1261] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [term 1 FOLLOWER]: Refusing update from remote peer 852838d44a2e4a838b2d5c1c2a3c5db5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:59:05.579702 5246 raft_consensus.cc:1261] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [term 1 FOLLOWER]: Refusing update from remote peer 852838d44a2e4a838b2d5c1c2a3c5db5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:59:05.580085 5337 consensus_queue.cc:1035] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:59:05.580263 4911 consensus_queue.cc:1035] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:59:05.583390 5339 mvcc.cc:205] Tried to move back new op lower bound from 7316368365891571712 to 7316368365829419008. Current Snapshot: MvccSnapshot[applied={T|T < 7316368365891571712 or (T in {7316368365891571712})}]
I20260808 20:59:05.585172 4639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:59:05.585278 4639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:59:05.586074 4639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Sending DeleteTablet for 3 replicas of tablet 81b81b5e8c314adebb8048bbc45742ba
I20260808 20:59:05.586378 4814 tablet_service.cc:1514] Processing DeleteTablet for tablet 81b81b5e8c314adebb8048bbc45742ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:05 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:56118
I20260808 20:59:05.586494 4915 tablet_replica.cc:330] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: stopping tablet replica
I20260808 20:59:05.586565 4915 raft_consensus.cc:2229] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:59:05.586531 5013 tablet_service.cc:1514] Processing DeleteTablet for tablet 81b81b5e8c314adebb8048bbc45742ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:05 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:45246
I20260808 20:59:05.586520 5215 tablet_service.cc:1514] Processing DeleteTablet for tablet 81b81b5e8c314adebb8048bbc45742ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:05 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:33000
I20260808 20:59:05.586648 4915 raft_consensus.cc:2258] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:05.586874 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 4695
I20260808 20:59:05.586891 5352 tablet_replica.cc:330] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: stopping tablet replica
I20260808 20:59:05.599620 5352 raft_consensus.cc:2229] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:59:05.599825 5352 raft_consensus.cc:2258] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:05.600571 5352 ts_tablet_manager.cc:1905] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:05.602514 5352 ts_tablet_manager.cc:1918] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:05.602633 5352 log.cc:1196] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/81b81b5e8c314adebb8048bbc45742ba
I20260808 20:59:05.603017 5352 ts_tablet_manager.cc:1939] T 81b81b5e8c314adebb8048bbc45742ba P 95181f5851114da09ff09791d505abd5: Deleting consensus metadata
I20260808 20:59:05.607769 4623 catalog_manager.cc:4841] TS 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785): tablet 81b81b5e8c314adebb8048bbc45742ba (table test-table [id=8846b7aace1045419a76ac0efd108cdf]) successfully deleted
I20260808 20:59:05.608031 5353 tablet_replica.cc:330] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: stopping tablet replica
I20260808 20:59:05.608220 5353 raft_consensus.cc:2229] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:59:05.608366 5353 raft_consensus.cc:2258] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:05.608989 5353 ts_tablet_manager.cc:1905] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:05.610576 5353 ts_tablet_manager.cc:1918] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:05.610661 5353 log.cc:1196] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/81b81b5e8c314adebb8048bbc45742ba
I20260808 20:59:05.610997 5353 ts_tablet_manager.cc:1939] T 81b81b5e8c314adebb8048bbc45742ba P 1057c3f7aa044c4ea15c8b023aea66a0: Deleting consensus metadata
I20260808 20:59:05.611505 4625 catalog_manager.cc:4841] TS 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967): tablet 81b81b5e8c314adebb8048bbc45742ba (table test-table [id=8846b7aace1045419a76ac0efd108cdf]) successfully deleted
W20260808 20:59:05.612974 4624 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:35965 (error 108)
I20260808 20:59:05.613448 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 4909
I20260808 20:59:05.613659 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:05.613480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35965 (local address 127.0.0.1:56136)
0808 20:59:05.613564 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.613618 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:35965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 20:59:05.617700 4624 catalog_manager.cc:4584] TS 852838d44a2e4a838b2d5c1c2a3c5db5 (127.15.232.129:35965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81b81b5e8c314adebb8048bbc45742ba: Network error: Client connection negotiation failed: client connection to 127.15.232.129:35965: connect: Connection refused (error 111)
I20260808 20:59:05.629987 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 5114
I20260808 20:59:05.642639 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 4597
I20260808 20:59:05.651875 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:38411
--webserver_interface=127.15.232.190
--webserver_port=33249
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:38411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:05.737447 5356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:05.737715 5356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:05.737758 5356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:05.739008 5356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:59:05.739079 5356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:05.739104 5356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:05.739125 5356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:59:05.739154 5356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:59:05.740780 5356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:38411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:38411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:05.741099 5356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:05.741356 5356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:05.746948 5364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.746948 5366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.747035 5363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:05.747224 5356 server_base.cc:1034] running on GCE node
I20260808 20:59:05.747531 5356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:05.747870 5356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:05.749058 5356 hybrid_clock.cc:648] HybridClock initialized: now 1786222745749038 us; error 42 us; skew 500 ppm
Aug 08 20:59:05 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:05.752054 5356 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:59:05.752596 5356 webserver.cc:460] Webserver started at http://127.15.232.190:33249/ using document root <none> and password file <none>
I20260808 20:59:05.752851 5356 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:05.752925 5356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:05.754416 5356 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 20:59:05.755275 5378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:05.755532 5356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:05.755657 5356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "db57001c48ac4b67891f56b4db0ca97a"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "20e8ddc02769ae6a25834668f2ee90a1"
server_key_iv: "cd093e8822e8a3944a68d854fb50993c"
server_key_version: "encryptionkey@0"
I20260808 20:59:05.756098 5356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:05.769129 5356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:05.771320 5356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:05.771571 5356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:05.776788 5356 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:38411
I20260808 20:59:05.776823 5450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:38411 every 8 connection(s)
I20260808 20:59:05.777231 5356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:59:05.778141 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 5356
I20260808 20:59:05.778489 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:35965
--local_ip_for_outbound_sockets=127.15.232.129
--tserver_master_addrs=127.15.232.190:38411
--webserver_port=36759
--webserver_interface=127.15.232.129
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:59:05.779590 5451 sys_catalog.cc:263] Verifying existing consensus state
I20260808 20:59:05.780694 5451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Bootstrap starting.
I20260808 20:59:05.789778 5451 log.cc:824] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:05.792500 5451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:59:05.792892 5451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a: Bootstrap complete.
I20260808 20:59:05.795320 5451 raft_consensus.cc:348] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:05.795782 5451 raft_consensus.cc:729] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: db57001c48ac4b67891f56b4db0ca97a, State: Initialized, Role: FOLLOWER
I20260808 20:59:05.796020 5451 consensus_queue.cc:260] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:05.796123 5451 raft_consensus.cc:388] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:59:05.796181 5451 raft_consensus.cc:482] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:59:05.796238 5451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 2 FOLLOWER]: Advancing to term 3
I20260808 20:59:05.797500 5451 raft_consensus.cc:504] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:05.797636 5451 leader_election.cc:302] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db57001c48ac4b67891f56b4db0ca97a; no voters:
I20260808 20:59:05.797870 5451 leader_election.cc:288] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 20:59:05.797920 5455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 3 FOLLOWER]: Leader election won for term 3
I20260808 20:59:05.798091 5455 raft_consensus.cc:686] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [term 3 LEADER]: Becoming Leader. State: Replica: db57001c48ac4b67891f56b4db0ca97a, State: Running, Role: LEADER
I20260808 20:59:05.798187 5451 sys_catalog.cc:564] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:59:05.798189 5455 consensus_queue.cc:237] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } }
I20260808 20:59:05.798612 5458 sys_catalog.cc:455] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "db57001c48ac4b67891f56b4db0ca97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } } }
I20260808 20:59:05.798628 5457 sys_catalog.cc:455] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: SysCatalogTable state changed. Reason: New leader db57001c48ac4b67891f56b4db0ca97a. Latest consensus state: current_term: 3 leader_uuid: "db57001c48ac4b67891f56b4db0ca97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db57001c48ac4b67891f56b4db0ca97a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 38411 } } }
I20260808 20:59:05.798840 5457 sys_catalog.cc:458] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:05.799026 5458 sys_catalog.cc:458] T 00000000000000000000000000000000 P db57001c48ac4b67891f56b4db0ca97a [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:05.799274 5466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:59:05.800321 5466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:59:05.800436 5466 catalog_manager.cc:1259] Loaded cluster ID: fa129c8b739d4574b543303e9980170b
I20260808 20:59:05.800477 5466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 20:59:05.800899 5466 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9dd521b9ad google::LogMessage::Fail() at ??:0
@ 0x7f9dd521e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f9dd521b4de google::LogMessage::Flush() at ??:0
@ 0x7f9dd521ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9ddd13c0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f9ddd1458b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9dd70e90d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9dd70de954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9dd7f426db start_thread at ??:0
@ 0x7f9dd3d1961f clone at ??:0
W20260808 20:59:05.904446 5453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:05.904781 5453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:05.904822 5453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:05.906725 5453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:05.906803 5453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:05.906849 5453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:59:05.909689 5453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:35965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=36759
--tserver_master_addrs=127.15.232.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:05.910028 5453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:05.910338 5453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:05.916105 5499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.916184 5502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:05.916098 5505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:05.916332 5453 server_base.cc:1034] running on GCE node
I20260808 20:59:05.916730 5453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:05.917042 5453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:05.918244 5453 hybrid_clock.cc:648] HybridClock initialized: now 1786222745918228 us; error 30 us; skew 500 ppm
Aug 08 20:59:05 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:05.921689 5453 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:59:05.922595 5453 webserver.cc:460] Webserver started at http://127.15.232.129:36759/ using document root <none> and password file <none>
I20260808 20:59:05.922915 5453 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:05.922988 5453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:05.925174 5453 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:05.926077 5515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:05.926290 5453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:05.926405 5453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "852838d44a2e4a838b2d5c1c2a3c5db5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "c449d90c5362de319eb05d4a9a232bc9"
server_key_iv: "6151816ae314024abb60ba001379ad19"
server_key_version: "encryptionkey@0"
I20260808 20:59:05.926931 5453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:05.948448 5453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:05.951267 5453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:05.951400 5453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:05.951884 5453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:05.952421 5525 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 20:59:05.954926 5453 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 20:59:05.955026 5453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 20:59:05.955112 5453 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 20:59:05.956002 5453 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 20:59:05.956070 5453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:05.956180 5525 tablet_bootstrap.cc:492] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Bootstrap starting.
I20260808 20:59:05.964893 5453 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:35965
I20260808 20:59:05.965348 5453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:59:05.965924 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 5453
I20260808 20:59:05.966262 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:32785
--local_ip_for_outbound_sockets=127.15.232.130
--tserver_master_addrs=127.15.232.190:38411
--webserver_port=42449
--webserver_interface=127.15.232.130
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:59:05.969229 5525 log.cc:824] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:05.969677 5667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:35965 every 8 connection(s)
I20260808 20:59:05.972309 5525 tablet_bootstrap.cc:492] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:59:05.972656 5525 tablet_bootstrap.cc:492] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Bootstrap complete.
I20260808 20:59:05.973837 5525 ts_tablet_manager.cc:1397] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.013s
I20260808 20:59:05.974995 5525 raft_consensus.cc:348] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.975265 5525 raft_consensus.cc:729] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852838d44a2e4a838b2d5c1c2a3c5db5, State: Initialized, Role: FOLLOWER
I20260808 20:59:05.975425 5525 consensus_queue.cc:260] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:05.975827 5525 ts_tablet_manager.cc:1428] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260808 20:59:06.057773 5671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:06.058009 5671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:06.058036 5671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:06.059271 5671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:06.059322 5671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:06.059351 5671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:59:06.060971 5671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:32785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=42449
--tserver_master_addrs=127.15.232.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:06.061249 5671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:06.061515 5671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:06.069569 5684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:06.069578 5683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:06.069579 5686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:06.070298 5671 server_base.cc:1034] running on GCE node
I20260808 20:59:06.070521 5671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:06.070825 5671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:06.071990 5671 hybrid_clock.cc:648] HybridClock initialized: now 1786222746071975 us; error 31 us; skew 500 ppm
W20260808 20:59:06.073007 5670 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 20:59:05.966325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:49611)
0808 20:59:05.966644 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:05.966702 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:05.967579 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:06.072782 (+105203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":139,"threads_started":1}
W20260808 20:59:06.073283 5668 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:38411 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.190:38411: Client connection negotiation failed: client connection to 127.15.232.190:38411: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 20:59:06.073678 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:06.073527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:47581)
0808 20:59:06.073601 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:06.073651 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 20:59:06.074198 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:06.074003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:35553)
0808 20:59:06.074105 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:06.074136 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260808 20:59:06.074373 5668 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 20:59:06 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:06.075074 5671 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:59:06.075620 5671 webserver.cc:460] Webserver started at http://127.15.232.130:42449/ using document root <none> and password file <none>
I20260808 20:59:06.075872 5671 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:06.075922 5671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:06.077327 5671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:59:06.078120 5700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:06.078336 5671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:59:06.078441 5671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "95181f5851114da09ff09791d505abd5"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "148f2775c79a49cdcb288f7cd5a273c5"
server_key_iv: "27a322db7a7d82aaf38c16629adca2c1"
server_key_version: "encryptionkey@0"
I20260808 20:59:06.078837 5671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:06.090559 5671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:06.092664 5671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:06.092839 5671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:06.093098 5671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:06.093425 5671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:06.093483 5671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:06.093529 5671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:06.093554 5671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:06.100253 5671 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:32785
I20260808 20:59:06.100381 5849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:32785 every 8 connection(s)
I20260808 20:59:06.100764 5671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:59:06.102488 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:06.102077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:55435)
0808 20:59:06.102251 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:06.102337 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
W20260808 20:59:06.102950 5851 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:38411 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.190:38411: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
I20260808 20:59:06.103264 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:06.103172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:54867)
0808 20:59:06.103218 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:06.103247 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 20:59:06.103618 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:06.103507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:49193)
0808 20:59:06.103578 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:06.103602 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 20:59:06.103750 5851 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 20:59:06.103974 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 5671
I20260808 20:59:06.104252 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:32967
--local_ip_for_outbound_sockets=127.15.232.131
--tserver_master_addrs=127.15.232.190:38411
--webserver_port=42569
--webserver_interface=127.15.232.131
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:06.189791 5856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:06.190008 5856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:06.190035 5856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:06.191265 5856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:06.191316 5856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:06.191344 5856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:59:06.192876 5856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:32967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=42569
--tserver_master_addrs=127.15.232.190:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:06.193133 5856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:06.193379 5856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:06.198951 5866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:06.198951 5862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:06.199165 5856 server_base.cc:1034] running on GCE node
W20260808 20:59:06.199033 5863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:06.199556 5856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:06.199852 5856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:06.201009 5856 hybrid_clock.cc:648] HybridClock initialized: now 1786222746201000 us; error 43 us; skew 500 ppm
Aug 08 20:59:06 dist-test-slave-c12x krb5kdc[3834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:06.203855 5856 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:59:06.204391 5856 webserver.cc:460] Webserver started at http://127.15.232.131:42569/ using document root <none> and password file <none>
I20260808 20:59:06.204638 5856 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:06.204694 5856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:06.206149 5856 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:06.206849 5875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:06.207047 5856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:06.207144 5856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "1057c3f7aa044c4ea15c8b023aea66a0"
format_stamp: "Formatted at 2026-08-08 20:59:04 on dist-test-slave-c12x"
server_key: "dfa8a8bcf346fae81de9d1e7462c518c"
server_key_iv: "9f09244945fde5296f03388388139ed9"
server_key_version: "encryptionkey@0"
I20260808 20:59:06.207579 5856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:06.244833 5856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:06.246973 5856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:06.247123 5856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:06.247396 5856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:06.247788 5856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:06.247845 5856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:06.247893 5856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:06.247920 5856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:06.254506 5856 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:32967
I20260808 20:59:06.254556 6036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:32967 every 8 connection(s)
I20260808 20:59:06.254937 5856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:59:06.256194 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:06.255672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:48715)
0808 20:59:06.255926 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:06.256046 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
W20260808 20:59:06.256657 6037 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:38411 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.190:38411: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
I20260808 20:59:06.257026 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:06.256926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:48053)
0808 20:59:06.256976 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:06.257007 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 20:59:06.257339 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:06.257261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:50729)
0808 20:59:06.257303 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:06.257325 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 20:59:06.257462 6037 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 20:59:06.260946 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 5856
I20260808 20:59:07.075150 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:07.074864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:40263)
0808 20:59:07.075045 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:07.075093 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":68,"threads_started":1}
I20260808 20:59:07.104322 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:07.104106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:46429)
0808 20:59:07.104224 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:07.104262 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260808 20:59:07.258121 6046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:07.257866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:39599)
0808 20:59:07.258025 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:07.258068 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260808 20:59:07.763638 6047 raft_consensus.cc:482] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:59:07.763775 6047 raft_consensus.cc:504] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:07.764204 6047 leader_election.cc:288] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785), 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967)
Aug 08 20:59:07 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.130@KRBTEST.COM
Aug 08 20:59:07 dist-test-slave-c12x krb5kdc[3834](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.131@KRBTEST.COM
I20260808 20:59:07.775951 6048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:07.764439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:32785 (local address 127.15.232.129:36369)
0808 20:59:07.764566 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:07.764580 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:07.764643 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:07.765522 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:07.765525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:07.765955 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:07.766164 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:07.766171 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:07.767358 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:07.767359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:07.768244 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:07.768247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:07.768938 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:07.768939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:07.769078 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:07.769785 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:07.769794 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:07.771507 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:07.773913 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:07.773921 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:07.773923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:07.774287 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:07.774590 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:07.774591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:07.774591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:07.774735 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:07.775294 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:07.775302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:07.775493 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 20:59:07.775620 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 20:59:07.775855 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":105,"thread_start_us":54,"threads_started":1}
I20260808 20:59:07.775969 6050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:07.764540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:49453 (local address 127.15.232.131:32967)
0808 20:59:07.764711 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:07.764723 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:07.764756 (+ 33us) server_negotiation.cc:407] Connection header received
0808 20:59:07.765222 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:07.765225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:07.765268 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:07.765476 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:07.766318 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:07.767206 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:07.768370 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:07.768817 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:07.768967 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:07.772031 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:07.772035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:07.772035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:07.772049 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:07.774071 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:07.774474 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:07.774477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:07.774477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:07.774609 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:07.774986 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:07.774987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:07.774987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:07.775231 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:07.775353 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:07.775688 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 20:59:07.775841 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":63,"threads_started":1}
I20260808 20:59:07.776216 6051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:07.764556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:36369 (local address 127.15.232.130:32785)
0808 20:59:07.764760 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:07.764768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:07.764786 (+ 18us) server_negotiation.cc:407] Connection header received
0808 20:59:07.765226 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:07.765228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:07.765268 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:07.765511 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:07.766321 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:07.767245 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:07.768367 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:07.768816 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:07.768968 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:07.771629 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:07.771634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:07.771634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:07.771655 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:07.773811 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:07.774386 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:07.774390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:07.774391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:07.774516 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:07.774841 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:07.774842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:07.774842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:07.775202 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:07.775277 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:07.775766 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 20:59:07.775965 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":83,"threads_started":1}
I20260808 20:59:07.776489 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:07.764399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:32967 (local address 127.15.232.129:49453)
0808 20:59:07.764559 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:07.764573 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:07.764640 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:07.765521 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:07.765524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:07.765954 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:07.766164 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:07.766171 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:07.767364 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:07.767365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:07.768243 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:07.768246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:07.768945 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:07.768946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:07.769081 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:07.769818 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:07.769823 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:07.771894 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:07.774161 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:07.774162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:07.774162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:07.774386 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:07.774726 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:07.774736 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:07.774736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:07.774876 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:07.775359 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:07.775360 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:07.775493 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 20:59:07.775622 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 20:59:07.775972 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":102,"thread_start_us":91,"threads_started":1}
W20260808 20:59:07.777107 5517 leader_election.cc:341] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785): Not found: Tablet not found: 81b81b5e8c314adebb8048bbc45742ba
W20260808 20:59:07.777271 5519 leader_election.cc:341] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967): Not found: Tablet not found: 81b81b5e8c314adebb8048bbc45742ba
I20260808 20:59:07.777309 5519 leader_election.cc:302] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852838d44a2e4a838b2d5c1c2a3c5db5; no voters: 1057c3f7aa044c4ea15c8b023aea66a0, 95181f5851114da09ff09791d505abd5
I20260808 20:59:07.777423 6047 raft_consensus.cc:2726] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 20:59:08.076150 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:08.075972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:57839)
0808 20:59:08.076056 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:08.076106 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 20:59:08.105192 6053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:08.104939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:51735)
0808 20:59:08.105081 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:08.105139 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260808 20:59:08.258939 6054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:08.258676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:58351)
0808 20:59:08.258831 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:08.258887 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260808 20:59:09.076952 6055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:09.076669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:53427)
0808 20:59:09.076843 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:09.076899 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
I20260808 20:59:09.106060 6056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:09.105799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:39585)
0808 20:59:09.105951 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:09.106010 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260808 20:59:09.259781 6057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:09.259542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:48623)
0808 20:59:09.259681 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:09.259732 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260808 20:59:10.016252 6058 raft_consensus.cc:482] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:59:10.016360 6058 raft_consensus.cc:504] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:10.016562 6058 leader_election.cc:288] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785), 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967)
W20260808 20:59:10.017023 5519 leader_election.cc:341] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967): Not found: Tablet not found: 81b81b5e8c314adebb8048bbc45742ba
W20260808 20:59:10.017104 5517 leader_election.cc:341] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785): Not found: Tablet not found: 81b81b5e8c314adebb8048bbc45742ba
I20260808 20:59:10.017211 5519 leader_election.cc:302] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852838d44a2e4a838b2d5c1c2a3c5db5; no voters: 1057c3f7aa044c4ea15c8b023aea66a0, 95181f5851114da09ff09791d505abd5
I20260808 20:59:10.017318 6058 raft_consensus.cc:2726] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 20:59:10.077734 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:10.077505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:48077)
0808 20:59:10.077631 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:10.077675 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260808 20:59:10.107023 6060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:10.106741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:45765)
0808 20:59:10.106896 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:10.106959 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260808 20:59:10.260514 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:10.260279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:40647)
0808 20:59:10.260408 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:10.260448 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260808 20:59:11.078723 6062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:11.078412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:56877)
0808 20:59:11.078615 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:11.078660 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":93,"threads_started":1}
I20260808 20:59:11.107877 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:11.107616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:60851)
0808 20:59:11.107782 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:11.107829 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260808 20:59:11.261288 6064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:11.261067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:41537)
0808 20:59:11.261202 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:11.261238 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260808 20:59:11.754259 6066 raft_consensus.cc:482] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:59:11.754372 6066 raft_consensus.cc:504] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:11.754609 6066 leader_election.cc:288] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785), 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967)
W20260808 20:59:11.755079 5517 leader_election.cc:341] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785): Not found: Tablet not found: 81b81b5e8c314adebb8048bbc45742ba
W20260808 20:59:11.755164 5519 leader_election.cc:341] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967): Not found: Tablet not found: 81b81b5e8c314adebb8048bbc45742ba
I20260808 20:59:11.755230 5519 leader_election.cc:302] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852838d44a2e4a838b2d5c1c2a3c5db5; no voters: 1057c3f7aa044c4ea15c8b023aea66a0, 95181f5851114da09ff09791d505abd5
I20260808 20:59:11.755324 6066 raft_consensus.cc:2726] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 20:59:12.079725 6067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:12.079377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:56075)
0808 20:59:12.079605 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:12.079656 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":156,"threads_started":1}
I20260808 20:59:12.108716 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:12.108477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:39823)
0808 20:59:12.108625 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:12.108666 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260808 20:59:12.262169 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:12.261915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:48865)
0808 20:59:12.262071 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:12.262118 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260808 20:59:13.080498 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:13.080257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:49819)
0808 20:59:13.080402 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:13.080447 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260808 20:59:13.109738 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:13.109423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:50311)
0808 20:59:13.109581 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:13.109624 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260808 20:59:13.263139 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:13.262817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:34813)
0808 20:59:13.263011 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:13.263065 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260808 20:59:14.081245 6075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:14.080996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:38357)
0808 20:59:14.081150 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:14.081195 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260808 20:59:14.110651 6077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:14.110416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:60985)
0808 20:59:14.110551 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:14.110603 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260808 20:59:14.264096 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:14.263797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:42153)
0808 20:59:14.263996 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:14.264044 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":96,"threads_started":1}
I20260808 20:59:15.082058 6079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:15.081778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:41593)
0808 20:59:15.081958 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:15.082008 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
I20260808 20:59:15.111897 6080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:15.111318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:46009)
0808 20:59:15.111604 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:15.111659 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":120,"threads_started":1}
I20260808 20:59:15.264945 6081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:15.264703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:33265)
0808 20:59:15.264856 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:15.264896 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260808 20:59:16.065660 6082 raft_consensus.cc:482] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:59:16.065773 6082 raft_consensus.cc:504] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95181f5851114da09ff09791d505abd5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 32785 } } peers { permanent_uuid: "1057c3f7aa044c4ea15c8b023aea66a0" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 32967 } } peers { permanent_uuid: "852838d44a2e4a838b2d5c1c2a3c5db5" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35965 } }
I20260808 20:59:16.065923 6082 leader_election.cc:288] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785), 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967)
W20260808 20:59:16.066344 5519 leader_election.cc:341] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1057c3f7aa044c4ea15c8b023aea66a0 (127.15.232.131:32967): Not found: Tablet not found: 81b81b5e8c314adebb8048bbc45742ba
W20260808 20:59:16.066480 5517 leader_election.cc:341] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 95181f5851114da09ff09791d505abd5 (127.15.232.130:32785): Not found: Tablet not found: 81b81b5e8c314adebb8048bbc45742ba
I20260808 20:59:16.066546 5517 leader_election.cc:302] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852838d44a2e4a838b2d5c1c2a3c5db5; no voters: 1057c3f7aa044c4ea15c8b023aea66a0, 95181f5851114da09ff09791d505abd5
I20260808 20:59:16.066641 6082 raft_consensus.cc:2726] T 81b81b5e8c314adebb8048bbc45742ba P 852838d44a2e4a838b2d5c1c2a3c5db5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 20:59:16.082818 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:16.082592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:54953)
0808 20:59:16.082734 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:16.082767 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260808 20:59:16.113179 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:16.112953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:60031)
0808 20:59:16.113093 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:16.113132 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260808 20:59:16.265764 6086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:16.265537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:59643)
0808 20:59:16.265674 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:16.265713 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260808 20:59:17.083576 6087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:17.083290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:34769)
0808 20:59:17.083476 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:17.083524 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260808 20:59:17.114140 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:17.113843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:58579)
0808 20:59:17.114049 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:17.114090 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":132,"threads_started":1}
I20260808 20:59:17.266562 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:17.266273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:46177)
0808 20:59:17.266465 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:17.266511 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260808 20:59:18.084446 6090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:18.084189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:37405)
0808 20:59:18.084356 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:18.084397 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260808 20:59:18.115038 6092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:18.114747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:50667)
0808 20:59:18.114946 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:18.114985 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260808 20:59:18.267416 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:18.267169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:43467)
0808 20:59:18.267322 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:18.267366 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260808 20:59:19.085321 6099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:19.085076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:47521)
0808 20:59:19.085234 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:19.085272 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260808 20:59:19.115943 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:19.115716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:51385)
0808 20:59:19.115844 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:19.115891 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260808 20:59:19.268260 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:19.268029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:47229)
0808 20:59:19.268170 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:19.268210 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260808 20:59:20.086162 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:20.085860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:55797)
0808 20:59:20.086054 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:20.086113 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
I20260808 20:59:20.116662 6103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:20.116416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:37859)
0808 20:59:20.116575 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:20.116613 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
I20260808 20:59:20.269172 6104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:20.268880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:43431)
0808 20:59:20.269053 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:20.269111 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":62,"threads_started":1}
I20260808 20:59:21.086994 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:21.086761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.129:58991)
0808 20:59:21.086896 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:21.086942 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260808 20:59:21.117635 6107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:21.117377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.130:38025)
0808 20:59:21.117536 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:21.117579 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260808 20:59:21.268788 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 5453
I20260808 20:59:21.276901 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:21.275413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:38411 (local address 127.15.232.131:51055)
0808 20:59:21.276816 (+ 1403us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:21.276853 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:38411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1377,"thread_start_us":1325,"threads_started":1}
I20260808 20:59:21.281448 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 5671
I20260808 20:59:21.292461 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 5856
2026-08-08T20:59:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18707 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 20:59:21 dist-test-slave-c12x krb5kdc[6111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 20:59:21 dist-test-slave-c12x krb5kdc[6111](info): set up 2 sockets
Aug 08 20:59:21 dist-test-slave-c12x krb5kdc[6111](info): commencing operation
krb5kdc: starting...
W20260808 20:59:22.532804 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 20:59:22 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T20:59:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T20:59:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 20:59:22.664045 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35921
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:35921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:22.756218 6134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:22.756488 6134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:22.756526 6134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:22.757712 6134 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 20:59:22.757777 6134 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 20:59:22.757802 6134 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 20:59:22.757830 6134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:59:22.757865 6134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:22.757894 6134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:22.757923 6134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:59:22.757951 6134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:59:22.759582 6134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:35921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:22.759886 6134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:22.760130 6134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 20:59:22.765682 6134 server_base.cc:1034] running on GCE node
W20260808 20:59:22.765758 6139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:22.765761 6142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:22.765702 6140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:22.766256 6134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:22.766582 6134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:22.767752 6134 hybrid_clock.cc:648] HybridClock initialized: now 1786222762767739 us; error 44 us; skew 500 ppm
Aug 08 20:59:22 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222762, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:22.774215 6134 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:59:22.774744 6134 webserver.cc:460] Webserver started at http://127.15.232.190:45383/ using document root <none> and password file <none>
I20260808 20:59:22.774993 6134 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:22.775064 6134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:22.775235 6134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:59:22.776473 6134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "735e09c669ca46898c5455e3f6633e06"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "4058403eb6b1c2d9c5bc23a91927ce9b"
server_key_iv: "8928b354102537c3dfdb6e255bdd7005"
server_key_version: "encryptionkey@0"
I20260808 20:59:22.776861 6134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "735e09c669ca46898c5455e3f6633e06"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "4058403eb6b1c2d9c5bc23a91927ce9b"
server_key_iv: "8928b354102537c3dfdb6e255bdd7005"
server_key_version: "encryptionkey@0"
I20260808 20:59:22.778333 6134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 20:59:22.779232 6152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:22.779531 6134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:22.779639 6134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "735e09c669ca46898c5455e3f6633e06"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "4058403eb6b1c2d9c5bc23a91927ce9b"
server_key_iv: "8928b354102537c3dfdb6e255bdd7005"
server_key_version: "encryptionkey@0"
I20260808 20:59:22.779724 6134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:22.800792 6134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:22.803148 6134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:22.803344 6134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:22.807842 6134 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:35921
I20260808 20:59:22.807874 6225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:35921 every 8 connection(s)
I20260808 20:59:22.808272 6134 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:59:22.809212 6228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:22.810760 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 6134
I20260808 20:59:22.810897 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 20:59:22.811136 16290 external_mini_cluster.cc:1428] Setting key 6a726a149c9be8f3ef960983330de4b1
I20260808 20:59:22.812565 6228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Bootstrap starting.
I20260808 20:59:22.813275 6228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:22.813652 6228 log.cc:824] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:22.814692 6228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: No bootstrap required, opened a new log
I20260808 20:59:22.817373 6228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 20:59:22.817574 6228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:22.817631 6228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735e09c669ca46898c5455e3f6633e06, State: Initialized, Role: FOLLOWER
I20260808 20:59:22.817981 6228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 20:59:22.818145 6228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:59:22.818269 6228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:59:22.818444 6228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 20:59:22 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:22.819311 6228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 20:59:22.819517 6228 leader_election.cc:302] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 735e09c669ca46898c5455e3f6633e06; no voters:
I20260808 20:59:22.819751 6228 leader_election.cc:288] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 20:59:22.819794 6233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:59:22.820000 6233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 1 LEADER]: Becoming Leader. State: Replica: 735e09c669ca46898c5455e3f6633e06, State: Running, Role: LEADER
I20260808 20:59:22.820147 6228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:59:22.820168 6233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 20:59:22.820703 6235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 735e09c669ca46898c5455e3f6633e06. Latest consensus state: current_term: 1 leader_uuid: "735e09c669ca46898c5455e3f6633e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } } }
I20260808 20:59:22.820909 6235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:22.820842 6234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "735e09c669ca46898c5455e3f6633e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } } }
I20260808 20:59:22.821161 6234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:22.821339 6246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:59:22.821916 6246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:59:22.823480 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:22.812025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34408 (local address 127.15.232.190:35921)
0808 20:59:22.812247 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:22.812255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:22.812284 (+ 29us) server_negotiation.cc:407] Connection header received
0808 20:59:22.813051 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:22.813060 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:22.813127 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:22.813255 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:22.814111 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:22.815091 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:22.815974 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:22.816431 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:22.816550 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:22.819391 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:22.819396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:22.819396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:22.819408 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:22.821731 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:22.822162 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:22.822165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:22.822166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:22.822274 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:22.822509 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:22.822510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:22.822511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:22.822778 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:22.822976 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:22.823049 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 20:59:22.823193 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":98,"threads_started":1}
I20260808 20:59:22.823541 6246 catalog_manager.cc:1347] Generated new cluster ID: 0b4a91612c374ed991e2d5b2aaf0b74c
I20260808 20:59:22.823602 6246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:59:22.832459 6246 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 20:59:22.833192 6246 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:59:22.840256 6246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Generated new TSK 0
I20260808 20:59:22.840468 6246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 20:59:22.875216 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35921
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:22.967392 6261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:22.967660 6261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:22.967689 6261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:22.968909 6261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:22.968956 6261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:22.968984 6261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:59:22.970533 6261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:22.970806 6261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:22.971112 6261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:22.976898 6267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:22.976877 6268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:22.976871 6271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:22.977156 6261 server_base.cc:1034] running on GCE node
I20260808 20:59:22.977401 6261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:22.977667 6261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:22.978848 6261 hybrid_clock.cc:648] HybridClock initialized: now 1786222762978823 us; error 37 us; skew 500 ppm
Aug 08 20:59:22 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222762, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:22.981837 6261 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:59:22.982281 6261 webserver.cc:460] Webserver started at http://127.15.232.129:34125/ using document root <none> and password file <none>
I20260808 20:59:22.982478 6261 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:22.982573 6261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:22.982693 6261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:59:22.983858 6261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "a9aee071485248359bdc6f81ec939873"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "3b19b9798c0f49f2da86f1ed8fe8217d"
server_key_iv: "d9b6c798764360b437186f6c28221305"
server_key_version: "encryptionkey@0"
I20260808 20:59:22.984184 6261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "a9aee071485248359bdc6f81ec939873"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "3b19b9798c0f49f2da86f1ed8fe8217d"
server_key_iv: "d9b6c798764360b437186f6c28221305"
server_key_version: "encryptionkey@0"
I20260808 20:59:22.985507 6261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 20:59:22.986258 6281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:22.986500 6261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 20:59:22.986608 6261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "a9aee071485248359bdc6f81ec939873"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "3b19b9798c0f49f2da86f1ed8fe8217d"
server_key_iv: "d9b6c798764360b437186f6c28221305"
server_key_version: "encryptionkey@0"
I20260808 20:59:22.986706 6261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:23.003392 6261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:23.005561 6261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:23.005728 6261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:23.005980 6261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:23.006306 6261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:23.006362 6261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:23.006404 6261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:23.006429 6261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:23.012682 6261 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:44565
I20260808 20:59:23.012831 6438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:44565 every 8 connection(s)
I20260808 20:59:23.013108 6261 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
Aug 08 20:59:23 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222762, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:23.021898 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 6261
I20260808 20:59:23.022030 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 20:59:23.022248 16290 external_mini_cluster.cc:1428] Setting key 11339353a62563d8f0acdbc7a5c20b57
I20260808 20:59:23.022879 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.013954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:45853 (local address 127.15.232.190:35921)
0808 20:59:23.014005 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.014007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.014187 (+ 180us) server_negotiation.cc:407] Connection header received
0808 20:59:23.014679 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.014686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.014697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.014758 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:23.015707 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.016250 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.017219 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.017418 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.017473 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.019657 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:23.019658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:23.019659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:23.019666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:23.021325 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:23.021715 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:23.021716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:23.021716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:23.021769 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:23.022133 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:23.022134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:23.022134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:23.022266 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:23.022345 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.022676 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.022779 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:59:23.023025 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.013887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:45853)
0808 20:59:23.014048 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:23.014102 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:23.014581 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:23.014811 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:23.014814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:23.015395 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:23.015624 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.015630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.016334 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.016335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.017141 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.017143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.017491 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.017492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.017599 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.018081 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:23.018089 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:23.019570 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:23.021400 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:23.021406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:23.021408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:23.021628 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:23.021832 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:23.021833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:23.021834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:23.021988 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:23.022350 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:23.022358 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:23.022500 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 20:59:23.022635 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 20:59:23.022781 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260808 20:59:23.024106 6439 heartbeater.cc:347] Connected to a master server at 127.15.232.190:35921
I20260808 20:59:23.024219 6439 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:23.024425 6439 heartbeater.cc:511] Master 127.15.232.190:35921 requested a full tablet report, sending...
I20260808 20:59:23.024998 6175 ts_manager.cc:194] Registered new tserver with Master: a9aee071485248359bdc6f81ec939873 (127.15.232.129:44565)
I20260808 20:59:23.025624 6175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:45853
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 20:59:23.050356 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35921
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:23.139257 6445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:23.139484 6445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:23.139509 6445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:23.140589 6445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:23.140625 6445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:23.140651 6445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:59:23.142030 6445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:23.142230 6445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:23.142403 6445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:23.147522 6455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:23.147589 6454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:23.147512 6458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:23.148001 6445 server_base.cc:1034] running on GCE node
I20260808 20:59:23.148229 6445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:23.148486 6445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:23.149662 6445 hybrid_clock.cc:648] HybridClock initialized: now 1786222763149653 us; error 36 us; skew 500 ppm
Aug 08 20:59:23 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222763, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:23.152257 6445 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:59:23.152671 6445 webserver.cc:460] Webserver started at http://127.15.232.130:38809/ using document root <none> and password file <none>
I20260808 20:59:23.152880 6445 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:23.152926 6445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:23.153016 6445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:59:23.154047 6445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "d50d2b86d95c4d68832444bfe4a8b872"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "b3a27b72170d2e63d2ec514cc76d7077"
server_key_iv: "65ac7882765552ab2c495cb79d1179c5"
server_key_version: "encryptionkey@0"
I20260808 20:59:23.154363 6445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "d50d2b86d95c4d68832444bfe4a8b872"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "b3a27b72170d2e63d2ec514cc76d7077"
server_key_iv: "65ac7882765552ab2c495cb79d1179c5"
server_key_version: "encryptionkey@0"
I20260808 20:59:23.155827 6445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 20:59:23.156709 6467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:23.156888 6445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:59:23.156984 6445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "d50d2b86d95c4d68832444bfe4a8b872"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "b3a27b72170d2e63d2ec514cc76d7077"
server_key_iv: "65ac7882765552ab2c495cb79d1179c5"
server_key_version: "encryptionkey@0"
I20260808 20:59:23.157047 6445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:23.176528 6445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:23.178988 6445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:23.179163 6445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:23.179422 6445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:23.179811 6445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:23.179864 6445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:23.179903 6445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:23.179929 6445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:23.186378 6445 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:41853
I20260808 20:59:23.186461 6643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:41853 every 8 connection(s)
I20260808 20:59:23.186808 6445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:59:23.187065 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 6445
I20260808 20:59:23.187177 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 20:59:23.187351 16290 external_mini_cluster.cc:1428] Setting key 998851583d270449f8c67b66ed475a5d
Aug 08 20:59:23 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222763, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
I20260808 20:59:23.197679 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.187698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:37035 (local address 127.15.232.190:35921)
0808 20:59:23.187753 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.187755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.187990 (+ 235us) server_negotiation.cc:407] Connection header received
0808 20:59:23.188519 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.188520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.188533 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.188596 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:23.189667 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.190301 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.191806 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.192132 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.192216 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.194483 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:23.194484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:23.194485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:23.194493 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:23.196183 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:23.196601 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:23.196602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:23.196603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:23.196664 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:23.196958 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:23.196960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:23.196960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:23.197112 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:23.197185 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.197462 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.197555 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 20:59:23.197854 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.187705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:37035)
0808 20:59:23.187866 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:23.187909 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:23.188416 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:23.188653 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:23.188657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:23.189290 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:23.189578 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.189585 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.190421 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.190422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.191717 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.191721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.192228 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.192229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.192339 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.192951 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:23.192962 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:23.194380 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:23.196282 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:23.196289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:23.196291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:23.196523 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:23.196764 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:23.196765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:23.196765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:23.196879 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:23.197181 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:23.197186 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:23.197311 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 20:59:23.197439 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 20:59:23.197613 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260808 20:59:23.198375 6644 heartbeater.cc:347] Connected to a master server at 127.15.232.190:35921
I20260808 20:59:23.198477 6644 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:23.198652 6644 heartbeater.cc:511] Master 127.15.232.190:35921 requested a full tablet report, sending...
I20260808 20:59:23.199071 6175 ts_manager.cc:194] Registered new tserver with Master: d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853)
I20260808 20:59:23.199572 6175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:37035
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 20:59:23.216522 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35921
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:23.306524 6657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:23.306807 6657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:23.306852 6657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:23.308127 6657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:23.308207 6657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:23.308256 6657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:59:23.310561 6657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:23.310913 6657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:23.311180 6657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:23.316779 6667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:23.316978 6657 server_base.cc:1034] running on GCE node
W20260808 20:59:23.316735 6665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:23.316794 6670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:23.317274 6657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:23.317600 6657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:23.318790 6657 hybrid_clock.cc:648] HybridClock initialized: now 1786222763318784 us; error 36 us; skew 500 ppm
Aug 08 20:59:23 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222763, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:23.321557 6657 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:59:23.322108 6657 webserver.cc:460] Webserver started at http://127.15.232.131:44671/ using document root <none> and password file <none>
I20260808 20:59:23.322376 6657 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:23.322450 6657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:23.322597 6657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 20:59:23.323977 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "a6328cca11e64cc5b610803f056c0da1"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "835e9b5b07be4c6b3f8f3ca0f17d6539"
server_key_iv: "8d23665e1b0a438c763efdfd4ffcb256"
server_key_version: "encryptionkey@0"
I20260808 20:59:23.324350 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "a6328cca11e64cc5b610803f056c0da1"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "835e9b5b07be4c6b3f8f3ca0f17d6539"
server_key_iv: "8d23665e1b0a438c763efdfd4ffcb256"
server_key_version: "encryptionkey@0"
I20260808 20:59:23.325943 6657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 20:59:23.327102 6680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:23.327385 6657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:23.327585 6657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "a6328cca11e64cc5b610803f056c0da1"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "835e9b5b07be4c6b3f8f3ca0f17d6539"
server_key_iv: "8d23665e1b0a438c763efdfd4ffcb256"
server_key_version: "encryptionkey@0"
I20260808 20:59:23.327811 6657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:23.347369 6657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:23.349572 6657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:23.349747 6657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:23.349994 6657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:23.350340 6657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:23.350402 6657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:23.350445 6657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:23.350477 6657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:23.357417 6657 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:42277
I20260808 20:59:23.357465 6847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:42277 every 8 connection(s)
I20260808 20:59:23.357815 6657 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:59:23.363374 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 6657
I20260808 20:59:23.363504 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 20:59:23.363701 16290 external_mini_cluster.cc:1428] Setting key a974b1712d94664115a5168adb574f13
Aug 08 20:59:23 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222763, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:23.368041 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.358610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:43701 (local address 127.15.232.190:35921)
0808 20:59:23.358656 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.358658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.358857 (+ 199us) server_negotiation.cc:407] Connection header received
0808 20:59:23.359415 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.359416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.359429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.359537 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:23.360620 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.361184 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.362214 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.362410 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.362484 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.365135 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:23.365136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:23.365137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:23.365144 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:23.366595 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:23.366973 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:23.366974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:23.366975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:23.367050 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:23.367274 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:23.367275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:23.367275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:23.367387 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:23.367553 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.367880 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.367964 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 20:59:23.368106 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.358542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:43701)
0808 20:59:23.358715 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:23.358763 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:23.359300 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:23.359620 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:23.359624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:23.360303 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:23.360504 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.360509 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.361274 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.361275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.362125 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.362128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.362493 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.362494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.362629 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.363142 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:23.363151 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:23.365050 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:23.366666 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:23.366672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:23.366673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:23.366888 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:23.367120 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:23.367120 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:23.367120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:23.367218 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:23.367536 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:23.367540 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:23.367684 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 20:59:23.367772 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 20:59:23.367904 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260808 20:59:23.368677 6848 heartbeater.cc:347] Connected to a master server at 127.15.232.190:35921
I20260808 20:59:23.368784 6848 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:23.368942 6848 heartbeater.cc:511] Master 127.15.232.190:35921 requested a full tablet report, sending...
I20260808 20:59:23.369314 6175 ts_manager.cc:194] Registered new tserver with Master: a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277)
I20260808 20:59:23.369701 6175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:43701
I20260808 20:59:23.375485 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:59:23.382987 6231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.376825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.15.232.190:35921)
0808 20:59:23.376897 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.376899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.376974 (+ 75us) server_negotiation.cc:407] Connection header received
0808 20:59:23.377018 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.377019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.377029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.377075 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:23.377688 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.378181 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.378870 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.379072 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.379129 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.380347 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:23.380348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:23.380349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:23.380356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:23.381727 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:23.382107 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:23.382108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:23.382108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:23.382159 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:23.382308 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:23.382308 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:23.382309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:23.382475 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:23.382551 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.382649 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.382862 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 20:59:23.384560 6175 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:59:23.386204 6175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:59:23.393138 6862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.388548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:42277 (local address 127.0.0.1:36968)
0808 20:59:23.388745 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:23.388787 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:23.388853 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:23.389429 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:23.389430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:23.389433 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:23.389484 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.389486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.390769 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.390770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.391840 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.391842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.392765 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.392766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.392869 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.392886 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 20:59:23.392970 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 20:59:23.393039 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":42,"thread_start_us":127,"threads_started":1}
I20260808 20:59:23.393141 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.388958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.15.232.131:42277)
0808 20:59:23.389150 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.389155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.389166 (+ 11us) server_negotiation.cc:407] Connection header received
0808 20:59:23.389208 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.389210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.389227 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.389351 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:23.389579 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.390662 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.391921 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.392655 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.392740 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.392804 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.393007 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.393051 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":107,"threads_started":1}
I20260808 20:59:23.393244 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.389284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.15.232.129:44565)
0808 20:59:23.389499 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.389504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.389579 (+ 75us) server_negotiation.cc:407] Connection header received
0808 20:59:23.389619 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.389621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.389636 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.389713 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:23.389829 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.390679 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.392114 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.392795 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.392872 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.392907 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.393097 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.393142 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":93,"threads_started":1}
I20260808 20:59:23.393532 6867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.388789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:44565 (local address 127.0.0.1:39756)
0808 20:59:23.388944 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:23.388954 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:23.388996 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:23.389711 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:23.389712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:23.389713 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:23.389772 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.389774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.390793 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.390794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.392017 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.392019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.392888 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.392889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.392951 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.392955 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:59:23.393008 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 20:59:23.393039 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":13,"thread_start_us":100,"threads_started":1}
I20260808 20:59:23.394253 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.388551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:41853 (local address 127.0.0.1:56272)
0808 20:59:23.388829 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:23.388840 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:23.388901 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:23.389142 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:23.389146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:23.389149 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:23.389222 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.389224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.390620 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.390621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.392013 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.392016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.394012 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.394014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.394079 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.394084 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:59:23.394138 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 20:59:23.394185 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":54,"thread_start_us":71,"threads_started":1}
I20260808 20:59:23.394287 6864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.388660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.15.232.130:41853)
0808 20:59:23.388824 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.388828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.388968 (+ 140us) server_negotiation.cc:407] Connection header received
0808 20:59:23.389015 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.389018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.389037 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.389133 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:23.389302 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.390522 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.392941 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.393940 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.394014 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.394074 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.394161 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.394203 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260808 20:59:23.394336 6742 tablet_service.cc:1467] Processing CreateTablet for tablet e7804873416041efbe1c78d6e1159e68 (DEFAULT_TABLE table=test-table [id=8be83d5f414f4a1e9f0f475dd00e324c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:23.394369 6348 tablet_service.cc:1467] Processing CreateTablet for tablet e7804873416041efbe1c78d6e1159e68 (DEFAULT_TABLE table=test-table [id=8be83d5f414f4a1e9f0f475dd00e324c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:23.394767 6348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7804873416041efbe1c78d6e1159e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:23.394767 6742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7804873416041efbe1c78d6e1159e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:23.395164 6539 tablet_service.cc:1467] Processing CreateTablet for tablet e7804873416041efbe1c78d6e1159e68 (DEFAULT_TABLE table=test-table [id=8be83d5f414f4a1e9f0f475dd00e324c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:23.395516 6539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7804873416041efbe1c78d6e1159e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:23.398250 6872 tablet_bootstrap.cc:492] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: Bootstrap starting.
I20260808 20:59:23.398741 6873 tablet_bootstrap.cc:492] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: Bootstrap starting.
I20260808 20:59:23.398880 6872 tablet_bootstrap.cc:654] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:23.398945 6870 tablet_bootstrap.cc:492] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Bootstrap starting.
I20260808 20:59:23.399405 6873 tablet_bootstrap.cc:654] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:23.399410 6872 log.cc:824] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:23.399628 6870 tablet_bootstrap.cc:654] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:23.400065 6873 log.cc:824] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:23.400240 6872 tablet_bootstrap.cc:492] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: No bootstrap required, opened a new log
I20260808 20:59:23.400290 6870 log.cc:824] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:23.400327 6872 ts_tablet_manager.cc:1397] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:59:23.400827 6873 tablet_bootstrap.cc:492] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: No bootstrap required, opened a new log
I20260808 20:59:23.400898 6873 ts_tablet_manager.cc:1397] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 20:59:23.401077 6870 tablet_bootstrap.cc:492] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: No bootstrap required, opened a new log
I20260808 20:59:23.401130 6870 ts_tablet_manager.cc:1397] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:59:23.402073 6872 raft_consensus.cc:348] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:23.402218 6872 raft_consensus.cc:374] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:23.402240 6872 raft_consensus.cc:729] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6328cca11e64cc5b610803f056c0da1, State: Initialized, Role: FOLLOWER
I20260808 20:59:23.402354 6872 consensus_queue.cc:260] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:23.402621 6872 ts_tablet_manager.cc:1428] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 20:59:23.402873 6873 raft_consensus.cc:348] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:23.403034 6873 raft_consensus.cc:374] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:23.403082 6873 raft_consensus.cc:729] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50d2b86d95c4d68832444bfe4a8b872, State: Initialized, Role: FOLLOWER
I20260808 20:59:23.403097 6870 raft_consensus.cc:348] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:23.403200 6873 consensus_queue.cc:260] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:23.403245 6870 raft_consensus.cc:374] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:23.403278 6870 raft_consensus.cc:729] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9aee071485248359bdc6f81ec939873, State: Initialized, Role: FOLLOWER
I20260808 20:59:23.403390 6870 consensus_queue.cc:260] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:23.403507 6873 ts_tablet_manager.cc:1428] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 20:59:23.403555 6644 heartbeater.cc:503] Master 127.15.232.190:35921 was elected leader, sending a full tablet report...
I20260808 20:59:23.403628 6848 heartbeater.cc:503] Master 127.15.232.190:35921 was elected leader, sending a full tablet report...
I20260808 20:59:23.403692 6870 ts_tablet_manager.cc:1428] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 20:59:23.403730 6439 heartbeater.cc:503] Master 127.15.232.190:35921 was elected leader, sending a full tablet report...
W20260808 20:59:23.437475 6645 tablet.cc:2369] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:59:23.511910 6879 raft_consensus.cc:482] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:59:23.512102 6879 raft_consensus.cc:504] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:23.512557 6879 leader_election.cc:288] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277), a9aee071485248359bdc6f81ec939873 (127.15.232.129:44565)
W20260808 20:59:23.513799 6440 tablet.cc:2369] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:59:23.516530 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.512799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:34663 (local address 127.15.232.131:42277)
0808 20:59:23.512849 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.512852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.512890 (+ 38us) server_negotiation.cc:407] Connection header received
0808 20:59:23.512937 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.512938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.512949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.513034 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:23.513207 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.514070 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.515482 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.516138 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.516217 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.516224 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.516412 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.516459 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 20:59:23.516548 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.512872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:48775 (local address 127.15.232.129:44565)
0808 20:59:23.512919 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.512921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.513220 (+ 299us) server_negotiation.cc:407] Connection header received
0808 20:59:23.513254 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.513255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.513268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.513327 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:23.513479 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.514218 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.515497 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.516131 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.516214 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.516224 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.516412 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.516486 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:59:23.516523 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.512725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:42277 (local address 127.15.232.130:34663)
0808 20:59:23.512774 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:23.512789 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:23.512839 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:23.513040 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:23.513041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:23.513043 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:23.513127 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.513129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.514160 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.514161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.515368 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.515370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.516225 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.516226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.516336 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.516339 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 20:59:23.516391 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 20:59:23.516431 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":25}
I20260808 20:59:23.516530 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.512765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:44565 (local address 127.15.232.130:48775)
0808 20:59:23.512936 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:23.512947 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:23.513004 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:23.513341 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:23.513342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:23.513343 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:23.513404 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.513406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.514292 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.514293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.515369 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:23.515371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.516227 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.516228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:23.516325 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.516329 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:59:23.516386 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 20:59:23.516431 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":4,"thread_start_us":141,"threads_started":1}
I20260808 20:59:23.516958 6776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7804873416041efbe1c78d6e1159e68" candidate_uuid: "d50d2b86d95c4d68832444bfe4a8b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6328cca11e64cc5b610803f056c0da1" is_pre_election: true
I20260808 20:59:23.516964 6373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7804873416041efbe1c78d6e1159e68" candidate_uuid: "d50d2b86d95c4d68832444bfe4a8b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9aee071485248359bdc6f81ec939873" is_pre_election: true
I20260808 20:59:23.517129 6776 raft_consensus.cc:2454] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50d2b86d95c4d68832444bfe4a8b872 in term 0.
I20260808 20:59:23.517136 6373 raft_consensus.cc:2454] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50d2b86d95c4d68832444bfe4a8b872 in term 0.
I20260808 20:59:23.517390 6471 leader_election.cc:302] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9aee071485248359bdc6f81ec939873, d50d2b86d95c4d68832444bfe4a8b872; no voters:
I20260808 20:59:23.517515 6879 raft_consensus.cc:2781] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:59:23.517573 6879 raft_consensus.cc:482] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:59:23.517598 6879 raft_consensus.cc:3037] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:23.518329 6879 raft_consensus.cc:504] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:23.518473 6879 leader_election.cc:288] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [CANDIDATE]: Term 1 election: Requested vote from peers a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277), a9aee071485248359bdc6f81ec939873 (127.15.232.129:44565)
I20260808 20:59:23.518620 6373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7804873416041efbe1c78d6e1159e68" candidate_uuid: "d50d2b86d95c4d68832444bfe4a8b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9aee071485248359bdc6f81ec939873"
I20260808 20:59:23.518657 6776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7804873416041efbe1c78d6e1159e68" candidate_uuid: "d50d2b86d95c4d68832444bfe4a8b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6328cca11e64cc5b610803f056c0da1"
I20260808 20:59:23.518714 6373 raft_consensus.cc:3037] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:23.518785 6776 raft_consensus.cc:3037] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:23.519474 6373 raft_consensus.cc:2454] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50d2b86d95c4d68832444bfe4a8b872 in term 1.
I20260808 20:59:23.519626 6471 leader_election.cc:302] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9aee071485248359bdc6f81ec939873, d50d2b86d95c4d68832444bfe4a8b872; no voters:
I20260808 20:59:23.519663 6776 raft_consensus.cc:2454] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50d2b86d95c4d68832444bfe4a8b872 in term 1.
I20260808 20:59:23.519755 6879 raft_consensus.cc:2781] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:59:23.519950 6879 raft_consensus.cc:686] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 1 LEADER]: Becoming Leader. State: Replica: d50d2b86d95c4d68832444bfe4a8b872, State: Running, Role: LEADER
I20260808 20:59:23.520076 6879 consensus_queue.cc:237] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:23.520974 6175 catalog_manager.cc:5495] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 reported cstate change: term changed from 0 to 1, leader changed from <none> to d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130). New cstate: current_term: 1 leader_uuid: "d50d2b86d95c4d68832444bfe4a8b872" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } health_report { overall_health: UNKNOWN } } }
I20260808 20:59:23.540674 6864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.538107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56284 (local address 127.15.232.130:41853)
0808 20:59:23.538174 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:23.538177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:23.538186 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:59:23.538234 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:23.538235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:23.538256 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:23.538334 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:59:23.538480 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.538979 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.539846 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:23.540049 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:23.540125 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:23.540296 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:59:23.540409 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:59:23.540507 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:23.540527 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 20:59:23.540558 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 20:59:23.542778 6776 raft_consensus.cc:1261] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [term 1 FOLLOWER]: Refusing update from remote peer d50d2b86d95c4d68832444bfe4a8b872: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:59:23.542790 6373 raft_consensus.cc:1261] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Refusing update from remote peer d50d2b86d95c4d68832444bfe4a8b872: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:59:23.543149 6879 consensus_queue.cc:1035] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:59:23.543314 6882 consensus_queue.cc:1035] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:59:23.545822 6885 mvcc.cc:205] Tried to move back new op lower bound from 7316368439467843584 to 7316368439378968576. Current Snapshot: MvccSnapshot[applied={T|T < 7316368439467843584}]
I20260808 20:59:23.546921 6886 mvcc.cc:205] Tried to move back new op lower bound from 7316368439467843584 to 7316368439378968576. Current Snapshot: MvccSnapshot[applied={T|T < 7316368439467843584 or (T in {7316368439467843584})}]
I20260808 20:59:23.547086 6887 mvcc.cc:205] Tried to move back new op lower bound from 7316368439467843584 to 7316368439378968576. Current Snapshot: MvccSnapshot[applied={T|T < 7316368439467843584}]
I20260808 20:59:23.548416 6174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:59:23.548528 6174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:59:23.549381 6174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Sending DeleteTablet for 3 replicas of tablet e7804873416041efbe1c78d6e1159e68
I20260808 20:59:23.549747 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 6261
I20260808 20:59:23.549738 6742 tablet_service.cc:1514] Processing DeleteTablet for tablet e7804873416041efbe1c78d6e1159e68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:23 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:36968
I20260808 20:59:23.549836 6348 tablet_service.cc:1514] Processing DeleteTablet for tablet e7804873416041efbe1c78d6e1159e68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:23 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:39756
I20260808 20:59:23.550282 6539 tablet_service.cc:1514] Processing DeleteTablet for tablet e7804873416041efbe1c78d6e1159e68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:23 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:56272
I20260808 20:59:23.552047 6897 tablet_replica.cc:330] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: stopping tablet replica
I20260808 20:59:23.552193 6897 raft_consensus.cc:2229] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:59:23.552229 6895 tablet_replica.cc:330] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: stopping tablet replica
I20260808 20:59:23.552395 6897 raft_consensus.cc:2258] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:23.552448 6895 raft_consensus.cc:2229] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:59:23.553304 6895 raft_consensus.cc:2258] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:23.553427 6897 ts_tablet_manager.cc:1905] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:23.555251 6895 ts_tablet_manager.cc:1905] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:23.555374 6897 ts_tablet_manager.cc:1918] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:23.555565 6897 log.cc:1196] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/e7804873416041efbe1c78d6e1159e68
I20260808 20:59:23.555927 6897 ts_tablet_manager.cc:1939] T e7804873416041efbe1c78d6e1159e68 P d50d2b86d95c4d68832444bfe4a8b872: Deleting consensus metadata
I20260808 20:59:23.556473 6156 catalog_manager.cc:4841] TS d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853): tablet e7804873416041efbe1c78d6e1159e68 (table test-table [id=8be83d5f414f4a1e9f0f475dd00e324c]) successfully deleted
I20260808 20:59:23.557600 6895 ts_tablet_manager.cc:1918] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:23.557703 6895 log.cc:1196] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/e7804873416041efbe1c78d6e1159e68
I20260808 20:59:23.558132 6895 ts_tablet_manager.cc:1939] T e7804873416041efbe1c78d6e1159e68 P a6328cca11e64cc5b610803f056c0da1: Deleting consensus metadata
I20260808 20:59:23.558720 6157 catalog_manager.cc:4841] TS a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277): tablet e7804873416041efbe1c78d6e1159e68 (table test-table [id=8be83d5f414f4a1e9f0f475dd00e324c]) successfully deleted
W20260808 20:59:23.567410 6156 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:44565 (error 108)
I20260808 20:59:23.567703 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 6445
I20260808 20:59:23.568044 6861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:23.567912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:44565 (local address 127.0.0.1:39770)
0808 20:59:23.567971 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:23.568002 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:44565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 20:59:23.568163 6156 catalog_manager.cc:4584] TS a9aee071485248359bdc6f81ec939873 (127.15.232.129:44565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7804873416041efbe1c78d6e1159e68: Network error: Client connection negotiation failed: client connection to 127.15.232.129:44565: connect: Connection refused (error 111)
I20260808 20:59:23.579041 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 6657
I20260808 20:59:23.591015 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 6134
I20260808 20:59:43.603782 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35921
--webserver_interface=127.15.232.190
--webserver_port=45383
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:35921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 20:59:43.706051 6899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:43.706347 6899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:43.706388 6899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:43.707698 6899 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 20:59:43.707767 6899 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 20:59:43.707789 6899 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 20:59:43.707818 6899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 20:59:43.707850 6899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:43.707878 6899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:43.707906 6899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 20:59:43.707929 6899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 20:59:43.709618 6899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:35921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=45383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:43.709975 6899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:43.710341 6899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:43.722810 6908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:43.722844 6899 server_base.cc:1034] running on GCE node
W20260808 20:59:43.722810 6910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:43.722887 6907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:43.723382 6899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:43.723779 6899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:43.724951 6899 hybrid_clock.cc:648] HybridClock initialized: now 1786222783724936 us; error 30 us; skew 500 ppm
Aug 08 20:59:43 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222783, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:43.730768 6899 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 20:59:43.731312 6899 webserver.cc:460] Webserver started at http://127.15.232.190:45383/ using document root <none> and password file <none>
I20260808 20:59:43.731617 6899 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:43.731685 6899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:43.733523 6899 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:43.734733 6921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:43.734973 6899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:43.735081 6899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "735e09c669ca46898c5455e3f6633e06"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "4058403eb6b1c2d9c5bc23a91927ce9b"
server_key_iv: "8928b354102537c3dfdb6e255bdd7005"
server_key_version: "encryptionkey@0"
I20260808 20:59:43.735584 6899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:43.749541 6899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:43.752367 6899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:43.752506 6899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:43.757651 6899 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:35921
I20260808 20:59:43.757782 6988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:35921 every 8 connection(s)
I20260808 20:59:43.758203 6899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 20:59:43.760030 6992 sys_catalog.cc:263] Verifying existing consensus state
I20260808 20:59:43.760744 6992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Bootstrap starting.
I20260808 20:59:43.761073 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 6899
I20260808 20:59:43.761435 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:44565
--local_ip_for_outbound_sockets=127.15.232.129
--tserver_master_addrs=127.15.232.190:35921
--webserver_port=34125
--webserver_interface=127.15.232.129
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:59:43.770452 6992 log.cc:824] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:43.773016 6992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:59:43.773391 6992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Bootstrap complete.
I20260808 20:59:43.776093 6992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 20:59:43.776361 6992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735e09c669ca46898c5455e3f6633e06, State: Initialized, Role: FOLLOWER
I20260808 20:59:43.776571 6992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 20:59:43.776674 6992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 20:59:43.776840 6992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 20:59:43.776928 6992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 1 FOLLOWER]: Advancing to term 2
I20260808 20:59:43.778093 6992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 20:59:43.778230 6992 leader_election.cc:302] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 735e09c669ca46898c5455e3f6633e06; no voters:
I20260808 20:59:43.778458 6992 leader_election.cc:288] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 20:59:43.778502 6998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 20:59:43.778640 6998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 2 LEADER]: Becoming Leader. State: Replica: 735e09c669ca46898c5455e3f6633e06, State: Running, Role: LEADER
I20260808 20:59:43.778764 6992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 20:59:43.778824 6998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 20:59:43.779198 6998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 735e09c669ca46898c5455e3f6633e06. Latest consensus state: current_term: 2 leader_uuid: "735e09c669ca46898c5455e3f6633e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } } }
I20260808 20:59:43.779199 7000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "735e09c669ca46898c5455e3f6633e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } } }
I20260808 20:59:43.779796 6998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:43.779832 7000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: This master's current role is: LEADER
I20260808 20:59:43.780680 7021 catalog_manager.cc:1259] Loaded cluster ID: 0b4a91612c374ed991e2d5b2aaf0b74c
I20260808 20:59:43.780752 7021 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: loading cluster ID for follower catalog manager: success
I20260808 20:59:43.781620 7021 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: acquiring CA information for follower catalog manager: success
I20260808 20:59:43.782018 7021 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 20:59:43.782112 7014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 20:59:43.782546 7014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 20:59:43.782630 7014 catalog_manager.cc:1259] Loaded cluster ID: 0b4a91612c374ed991e2d5b2aaf0b74c
I20260808 20:59:43.782661 7014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 20:59:43.782837 7014 catalog_manager.cc:1504] Loading token signing keys...
I20260808 20:59:43.782919 7014 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Loaded TSK: 0
I20260808 20:59:43.799741 7014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Generated new TSK 1
I20260808 20:59:43.799889 7014 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 20:59:43.863229 6995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:43.863559 6995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:43.863592 6995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:43.865034 6995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:43.865095 6995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:43.865127 6995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 20:59:43.866782 6995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:44565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=34125
--tserver_master_addrs=127.15.232.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:43.867079 6995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:43.867344 6995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:43.872947 7028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:43.872983 7029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:43.873088 6995 server_base.cc:1034] running on GCE node
W20260808 20:59:43.873132 7031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:43.873456 6995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:43.873769 6995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:43.874989 6995 hybrid_clock.cc:648] HybridClock initialized: now 1786222783874965 us; error 33 us; skew 500 ppm
Aug 08 20:59:43 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222783, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:43.878610 6995 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 20:59:43.879273 6995 webserver.cc:460] Webserver started at http://127.15.232.129:34125/ using document root <none> and password file <none>
I20260808 20:59:43.879586 6995 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:43.879731 6995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:43.881922 6995 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 20:59:43.882754 7040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:43.883025 6995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:59:43.883147 6995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "a9aee071485248359bdc6f81ec939873"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "3b19b9798c0f49f2da86f1ed8fe8217d"
server_key_iv: "d9b6c798764360b437186f6c28221305"
server_key_version: "encryptionkey@0"
I20260808 20:59:43.883594 6995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:43.898802 6995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:43.901530 6995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:43.901961 6995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:43.902544 6995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:43.903664 7048 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 20:59:43.905390 6995 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 20:59:43.905459 6995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.001s
I20260808 20:59:43.905488 6995 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 20:59:43.906221 6995 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 20:59:43.906268 6995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 20:59:43.906319 7048 tablet_bootstrap.cc:492] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Bootstrap starting.
I20260808 20:59:43.914453 6995 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:44565
I20260808 20:59:43.914541 7206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:44565 every 8 connection(s)
I20260808 20:59:43.915067 6995 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 20:59:43.918380 7048 log.cc:824] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:43.919082 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 6995
I20260808 20:59:43.919420 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:41853
--local_ip_for_outbound_sockets=127.15.232.130
--tserver_master_addrs=127.15.232.190:35921
--webserver_port=38809
--webserver_interface=127.15.232.130
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 20:59:43.922757 7048 tablet_bootstrap.cc:492] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 20:59:43.923146 7048 tablet_bootstrap.cc:492] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Bootstrap complete.
I20260808 20:59:43.924482 7048 ts_tablet_manager.cc:1397] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.007s
Aug 08 20:59:43 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222783, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:43.925845 7048 raft_consensus.cc:348] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:43.926196 7048 raft_consensus.cc:729] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9aee071485248359bdc6f81ec939873, State: Initialized, Role: FOLLOWER
I20260808 20:59:43.926394 7048 consensus_queue.cc:260] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:43.926679 7048 ts_tablet_manager.cc:1428] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 20:59:43.930864 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:43.915941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:40123)
0808 20:59:43.916280 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:43.916337 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:43.916996 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:43.917680 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:43.917686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:43.918104 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:43.918402 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:43.918411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:43.920609 (+ 2198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:43.920611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:43.922012 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:43.922016 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:43.923027 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:43.923028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:43.923179 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:43.923847 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:43.923868 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:43.925897 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:43.928616 (+ 2719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:43.928626 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:43.928629 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:43.928979 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:43.929327 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:43.929328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:43.929329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:43.929490 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:43.930047 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:43.930055 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:43.930239 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 20:59:43.930361 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 20:59:43.930582 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":114,"threads_started":1}
I20260808 20:59:43.931753 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:43.916047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:40123 (local address 127.15.232.190:35921)
0808 20:59:43.917208 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:43.917217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:43.917254 (+ 37us) server_negotiation.cc:407] Connection header received
0808 20:59:43.917388 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:43.917393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:43.917447 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:43.917593 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:43.918576 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:43.920453 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:43.922135 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:43.922784 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:43.922899 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:43.926039 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:43.926044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:43.926045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:43.926059 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:43.928503 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:43.929087 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:43.929091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:43.929092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:43.929240 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:43.929576 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:43.929577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:43.929578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:43.929925 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:43.930014 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:43.931240 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0808 20:59:43.931411 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1108,"thread_start_us":90,"threads_started":1}
I20260808 20:59:43.932418 7207 heartbeater.cc:347] Connected to a master server at 127.15.232.190:35921
I20260808 20:59:43.932530 7207 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:43.932747 7207 heartbeater.cc:511] Master 127.15.232.190:35921 requested a full tablet report, sending...
I20260808 20:59:43.933604 6941 ts_manager.cc:194] Registered new tserver with Master: a9aee071485248359bdc6f81ec939873 (127.15.232.129:44565)
I20260808 20:59:43.934979 6941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:40123
I20260808 20:59:43.935992 7207 heartbeater.cc:503] Master 127.15.232.190:35921 was elected leader, sending a full tablet report...
Aug 08 20:59:43 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222783, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.129@KRBTEST.COM
I20260808 20:59:43.943147 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:43.934424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55622 (local address 127.15.232.129:44565)
0808 20:59:43.934619 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:43.934623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:43.934797 (+ 174us) server_negotiation.cc:407] Connection header received
0808 20:59:43.934918 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:43.934921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:43.934940 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:43.935038 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:43.935645 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:43.936769 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:43.937510 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:43.937961 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:43.938042 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:43.940350 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:43.940358 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:43.940358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:43.940365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:43.941891 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:43.942353 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:43.942356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:43.942356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:43.942411 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:43.942612 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:43.942613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:43.942613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:43.942757 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:43.942821 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:43.942998 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 20:59:43.943078 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":85,"threads_started":1}
I20260808 20:59:43.943207 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:43.934358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:44565 (local address 127.0.0.1:55622)
0808 20:59:43.934680 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:43.934709 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:43.934810 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:43.935048 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:43.935054 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:43.935491 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:43.935556 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:43.935559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:43.936848 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:43.936849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:43.937430 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:43.937432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:43.938048 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:43.938049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:43.938106 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:43.938660 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:43.938664 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:43.940256 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:43.941978 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:43.941985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:43.941987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:43.942256 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:43.942477 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:43.942478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:43.942478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:43.942549 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:43.942819 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:43.942826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:43.942883 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 20:59:43.942977 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 20:59:43.943101 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":111,"threads_started":1}
I20260808 20:59:43.943617 7113 tablet_service.cc:1514] Processing DeleteTablet for tablet e7804873416041efbe1c78d6e1159e68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:23 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:55622
I20260808 20:59:43.944232 7219 tablet_replica.cc:330] stopping tablet replica
I20260808 20:59:43.944355 7219 raft_consensus.cc:2229] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:59:43.944451 7219 raft_consensus.cc:2258] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:43.944779 7219 ts_tablet_manager.cc:1905] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:43.947117 7219 ts_tablet_manager.cc:1918] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:43.947221 7219 log.cc:1196] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal/wals/e7804873416041efbe1c78d6e1159e68
I20260808 20:59:43.947487 7219 ts_tablet_manager.cc:1939] T e7804873416041efbe1c78d6e1159e68 P a9aee071485248359bdc6f81ec939873: Deleting consensus metadata
I20260808 20:59:43.948066 6924 catalog_manager.cc:4841] TS a9aee071485248359bdc6f81ec939873 (127.15.232.129:44565): tablet e7804873416041efbe1c78d6e1159e68 (table test-table [id=8be83d5f414f4a1e9f0f475dd00e324c]) successfully deleted
W20260808 20:59:44.016182 7215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:44.016419 7215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:44.016444 7215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:44.017741 7215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:44.017794 7215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:44.017822 7215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 20:59:44.019304 7215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:41853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=38809
--tserver_master_addrs=127.15.232.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:44.019593 7215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:44.019938 7215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:44.025977 7230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:44.025946 7225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:44.026050 7215 server_base.cc:1034] running on GCE node
W20260808 20:59:44.026255 7226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:44.026485 7215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:44.026764 7215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:44.027951 7215 hybrid_clock.cc:648] HybridClock initialized: now 1786222784027945 us; error 41 us; skew 500 ppm
Aug 08 20:59:44 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222784, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:44.030756 7215 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 20:59:44.031248 7215 webserver.cc:460] Webserver started at http://127.15.232.130:38809/ using document root <none> and password file <none>
I20260808 20:59:44.031564 7215 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:44.031656 7215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:44.033264 7215 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:59:44.034071 7238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:44.034279 7215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 20:59:44.034428 7215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "d50d2b86d95c4d68832444bfe4a8b872"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "b3a27b72170d2e63d2ec514cc76d7077"
server_key_iv: "65ac7882765552ab2c495cb79d1179c5"
server_key_version: "encryptionkey@0"
I20260808 20:59:44.034847 7215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:44.055300 7215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:44.058303 7215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:44.058511 7215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:44.058974 7215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:44.059378 7215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:44.059441 7215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:44.059533 7215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:44.059556 7215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:44.066615 7215 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:41853
I20260808 20:59:44.066655 7404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:41853 every 8 connection(s)
I20260808 20:59:44.067070 7215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 20:59:44.068156 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 7215
I20260808 20:59:44.068524 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:42277
--local_ip_for_outbound_sockets=127.15.232.131
--tserver_master_addrs=127.15.232.190:35921
--webserver_port=44671
--webserver_interface=127.15.232.131
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 20:59:44 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222784, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:44.078361 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.067978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:36101 (local address 127.15.232.190:35921)
0808 20:59:44.068025 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.068027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.068438 (+ 411us) server_negotiation.cc:407] Connection header received
0808 20:59:44.069176 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.069178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.069198 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.069286 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:44.070277 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.070795 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.071925 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.072150 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.072231 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.074588 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:44.074590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:44.074590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:44.074598 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:44.076509 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:44.077151 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:44.077153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:44.077153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:44.077213 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:44.077581 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:44.077582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:44.077583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:44.077710 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:44.077780 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:44.078195 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 20:59:44.078288 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 20:59:44.078687 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.068021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:36101)
0808 20:59:44.068277 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:44.068333 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:44.069027 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:44.069340 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:44.069344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:44.069962 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:44.070168 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.070173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.070901 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.070902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.071822 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.071825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.072236 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.072238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.072376 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.073037 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:44.073046 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:44.074468 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:44.076638 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:44.076649 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:44.076651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:44.077058 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:44.077299 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:44.077300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:44.077300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:44.077479 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:44.077796 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:44.077804 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:44.078003 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 20:59:44.078152 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 20:59:44.078384 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":107,"threads_started":1}
I20260808 20:59:44.079459 7407 heartbeater.cc:347] Connected to a master server at 127.15.232.190:35921
I20260808 20:59:44.079617 7407 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:44.079861 7407 heartbeater.cc:511] Master 127.15.232.190:35921 requested a full tablet report, sending...
I20260808 20:59:44.080422 6941 ts_manager.cc:194] Registered new tserver with Master: d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853)
I20260808 20:59:44.080894 6941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:36101
W20260808 20:59:44.156960 7410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 20:59:44.157176 7410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 20:59:44.157200 7410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 20:59:44.158303 7410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 20:59:44.158339 7410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 20:59:44.158366 7410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 20:59:44.159845 7410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:42277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=44671
--tserver_master_addrs=127.15.232.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 20:59:44.160043 7410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 20:59:44.160224 7410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 20:59:44.165802 7416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 20:59:44.165879 7415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:44.165925 7410 server_base.cc:1034] running on GCE node
W20260808 20:59:44.166123 7419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 20:59:44.166343 7410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 20:59:44.166606 7410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 20:59:44.167769 7410 hybrid_clock.cc:648] HybridClock initialized: now 1786222784167759 us; error 32 us; skew 500 ppm
Aug 08 20:59:44 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222784, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 20:59:44.170498 7410 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 20:59:44.171159 7410 webserver.cc:460] Webserver started at http://127.15.232.131:44671/ using document root <none> and password file <none>
I20260808 20:59:44.171415 7410 fs_manager.cc:359] Metadata directory not provided
I20260808 20:59:44.171572 7410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 20:59:44.173125 7410 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:44.173889 7429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:44.174106 7410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 20:59:44.174214 7410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "a6328cca11e64cc5b610803f056c0da1"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "835e9b5b07be4c6b3f8f3ca0f17d6539"
server_key_iv: "8d23665e1b0a438c763efdfd4ffcb256"
server_key_version: "encryptionkey@0"
I20260808 20:59:44.174656 7410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 20:59:44.191215 7410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 20:59:44.193828 7410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 20:59:44.194013 7410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 20:59:44.194293 7410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 20:59:44.194636 7410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 20:59:44.194694 7410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:44.194730 7410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 20:59:44.194777 7410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 20:59:44.200870 7410 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:42277
I20260808 20:59:44.200920 7592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:42277 every 8 connection(s)
I20260808 20:59:44.201332 7410 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 20:59:44.205204 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 7410
Aug 08 20:59:44 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222784, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 20:59:44.212795 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.202174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:60095 (local address 127.15.232.190:35921)
0808 20:59:44.202266 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.202268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.202483 (+ 215us) server_negotiation.cc:407] Connection header received
0808 20:59:44.203056 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.203057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.203070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.203124 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:44.204170 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.204674 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.206237 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.206426 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.206477 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.209382 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:44.209384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:44.209384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:44.209392 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:44.210896 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:44.211423 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:44.211424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:44.211424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:44.211571 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:44.211906 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:44.211908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:44.211908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:44.212087 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:44.212171 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:44.212595 (+ 424us) server_negotiation.cc:299] Negotiation successful
0808 20:59:44.212711 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 20:59:44.213112 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.202138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:60095)
0808 20:59:44.202349 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:44.202387 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:44.202926 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:44.203217 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:44.203220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:44.203859 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 20:59:44.204062 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.204067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.204777 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.204778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.206159 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.206162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.206547 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.206549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.206712 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.207341 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 20:59:44.207350 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 20:59:44.209283 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 20:59:44.210993 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:44.211001 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:44.211004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:44.211339 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:44.211647 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 20:59:44.211648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 20:59:44.211648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 20:59:44.211804 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 20:59:44.212172 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 20:59:44.212178 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 20:59:44.212311 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 20:59:44.212445 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 20:59:44.212637 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260808 20:59:44.213672 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.205662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.15.232.190:35921)
0808 20:59:44.205791 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.205792 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.205956 (+ 164us) server_negotiation.cc:407] Connection header received
0808 20:59:44.205980 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.205981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.205992 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.206051 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:44.206827 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.207596 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.208541 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.208885 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.208957 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.210236 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:44.210237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:44.210238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:44.210245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:44.212012 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:44.212399 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:44.212400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:44.212401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:44.212465 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:44.212847 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:44.212849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:44.212849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:44.213012 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:44.213441 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:44.213489 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 20:59:44.213597 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260808 20:59:44.213686 7594 heartbeater.cc:347] Connected to a master server at 127.15.232.190:35921
I20260808 20:59:44.213801 7594 heartbeater.cc:464] Registering TS with master...
I20260808 20:59:44.214016 7594 heartbeater.cc:511] Master 127.15.232.190:35921 requested a full tablet report, sending...
I20260808 20:59:44.214437 6941 ts_manager.cc:194] Registered new tserver with Master: a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277)
I20260808 20:59:44.214787 6941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:60095
I20260808 20:59:44.225443 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 20:59:44.228585 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.225852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40556 (local address 127.15.232.190:35921)
0808 20:59:44.225927 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.225929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.226049 (+ 120us) server_negotiation.cc:407] Connection header received
0808 20:59:44.226116 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.226116 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.226159 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.226246 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:59:44.226364 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.226881 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.227805 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.228034 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.228107 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.228314 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:59:44.228369 (+ 55us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 20:59:44.228528 (+ 159us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40556: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 20:59:44.228683 7600 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40556: token expired
W20260808 20:59:44.228904 6854 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.232.190:35921)
I20260808 20:59:44.236110 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.229246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.15.232.190:35921)
0808 20:59:44.229313 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.229314 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.229321 (+ 7us) server_negotiation.cc:407] Connection header received
0808 20:59:44.229341 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.229342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.229350 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.229397 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 20:59:44.229908 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.230364 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.231098 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.231279 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.231339 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.232769 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 20:59:44.232771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 20:59:44.232771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 20:59:44.232779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 20:59:44.234675 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:44.235134 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:44.235135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:44.235135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:44.235200 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 20:59:44.235509 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 20:59:44.235510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 20:59:44.235511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 20:59:44.235640 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 20:59:44.235745 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:44.235922 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 20:59:44.235999 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 20:59:44.237591 6941 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 20:59:44.239573 6941 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 20:59:44.242108 7113 tablet_service.cc:1467] Processing CreateTablet for tablet dfa2abf9939045fea80c1390a26d1d67 (DEFAULT_TABLE table=test-table [id=16ce24b11ec244548c65297d4e498354]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:44.242345 7113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfa2abf9939045fea80c1390a26d1d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:44.243979 7048 tablet_bootstrap.cc:492] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Bootstrap starting.
I20260808 20:59:44.244362 7048 tablet_bootstrap.cc:654] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:44.244976 7048 tablet_bootstrap.cc:492] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: No bootstrap required, opened a new log
I20260808 20:59:44.245036 7048 ts_tablet_manager.cc:1397] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 20:59:44.245394 7048 raft_consensus.cc:348] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:44.245476 7048 raft_consensus.cc:374] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:44.245509 7048 raft_consensus.cc:729] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9aee071485248359bdc6f81ec939873, State: Initialized, Role: FOLLOWER
I20260808 20:59:44.245571 7048 consensus_queue.cc:260] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:44.245674 7048 ts_tablet_manager.cc:1428] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 20:59:44.246503 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.241757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:41853 (local address 127.0.0.1:60134)
0808 20:59:44.241821 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:44.241835 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:44.241883 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:44.242336 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:44.242338 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:44.242340 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:44.242398 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.242402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.243854 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.243855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.245059 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.245063 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.246189 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.246190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.246257 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.246276 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 20:59:44.246370 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 20:59:44.246432 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 20:59:44.246891 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.241916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.15.232.130:41853)
0808 20:59:44.242127 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.242139 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.242149 (+ 10us) server_negotiation.cc:407] Connection header received
0808 20:59:44.242185 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.242186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.242200 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.242550 (+ 350us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:44.242639 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.243745 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.245241 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.246077 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.246166 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.246226 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:44.246760 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 20:59:44.246806 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":101,"threads_started":1}
I20260808 20:59:44.247689 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.243559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.15.232.131:42277)
0808 20:59:44.243778 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.243784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.243796 (+ 12us) server_negotiation.cc:407] Connection header received
0808 20:59:44.243838 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.243840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.243858 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.243950 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:44.244143 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.245242 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.246590 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.247307 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.247374 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.247419 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:44.247598 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 20:59:44.247630 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":106,"threads_started":1}
I20260808 20:59:44.247747 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.241826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:42277 (local address 127.0.0.1:40540)
0808 20:59:44.242110 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:44.242122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:44.242180 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:44.243949 (+ 1769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:44.243951 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:44.243953 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:44.244002 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.244004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.245325 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.245326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.246490 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.246492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.247399 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.247400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.247508 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.247513 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 20:59:44.247588 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 20:59:44.247636 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260808 20:59:44.247701 7307 tablet_service.cc:1467] Processing CreateTablet for tablet dfa2abf9939045fea80c1390a26d1d67 (DEFAULT_TABLE table=test-table [id=16ce24b11ec244548c65297d4e498354]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:44.248111 7307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfa2abf9939045fea80c1390a26d1d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:44.248375 7505 tablet_service.cc:1467] Processing CreateTablet for tablet dfa2abf9939045fea80c1390a26d1d67 (DEFAULT_TABLE table=test-table [id=16ce24b11ec244548c65297d4e498354]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 20:59:44.248615 7505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfa2abf9939045fea80c1390a26d1d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 20:59:44.251268 7607 tablet_bootstrap.cc:492] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: Bootstrap starting.
I20260808 20:59:44.251574 7608 tablet_bootstrap.cc:492] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: Bootstrap starting.
I20260808 20:59:44.251955 7607 tablet_bootstrap.cc:654] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:44.252312 7608 tablet_bootstrap.cc:654] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: Neither blocks nor log segments found. Creating new log.
I20260808 20:59:44.252609 7607 log.cc:824] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:44.252954 7608 log.cc:824] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: Log is configured to *not* fsync() on all Append() calls
I20260808 20:59:44.253844 7607 tablet_bootstrap.cc:492] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: No bootstrap required, opened a new log
I20260808 20:59:44.253865 7608 tablet_bootstrap.cc:492] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: No bootstrap required, opened a new log
I20260808 20:59:44.253948 7607 ts_tablet_manager.cc:1397] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 20:59:44.253939 7608 ts_tablet_manager.cc:1397] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 20:59:44.255355 7607 raft_consensus.cc:348] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:44.255508 7608 raft_consensus.cc:348] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:44.255630 7607 raft_consensus.cc:374] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:44.255679 7608 raft_consensus.cc:374] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 20:59:44.255690 7607 raft_consensus.cc:729] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50d2b86d95c4d68832444bfe4a8b872, State: Initialized, Role: FOLLOWER
I20260808 20:59:44.255730 7608 raft_consensus.cc:729] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6328cca11e64cc5b610803f056c0da1, State: Initialized, Role: FOLLOWER
I20260808 20:59:44.255841 7607 consensus_queue.cc:260] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:44.255908 7608 consensus_queue.cc:260] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:44.256172 7607 ts_tablet_manager.cc:1428] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 20:59:44.256254 7608 ts_tablet_manager.cc:1428] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 20:59:44.256254 7594 heartbeater.cc:503] Master 127.15.232.190:35921 was elected leader, sending a full tablet report...
I20260808 20:59:44.256325 7407 heartbeater.cc:503] Master 127.15.232.190:35921 was elected leader, sending a full tablet report...
I20260808 20:59:44.312740 7216 raft_consensus.cc:482] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 20:59:44.312992 7216 raft_consensus.cc:504] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:44.313473 7216 leader_election.cc:288] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853), a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277)
I20260808 20:59:44.316890 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.313709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:41853 (local address 127.15.232.129:42563)
0808 20:59:44.313777 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:44.313790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:44.313857 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:44.314048 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:44.314049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:44.314051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:44.314140 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.314142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.314877 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.314878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.315844 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.315847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.316563 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.316565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.316649 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.316670 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 20:59:44.316747 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 20:59:44.316801 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 20:59:44.316900 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.313793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:42563 (local address 127.15.232.130:41853)
0808 20:59:44.313842 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.313845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.313858 (+ 13us) server_negotiation.cc:407] Connection header received
0808 20:59:44.313942 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.313943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.313958 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.314042 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:44.314226 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.314766 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.316053 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.316456 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.316563 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.316573 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:44.316755 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 20:59:44.316800 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 20:59:44.317207 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.313811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:39207 (local address 127.15.232.131:42277)
0808 20:59:44.313868 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.313870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.314034 (+ 164us) server_negotiation.cc:407] Connection header received
0808 20:59:44.314122 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.314123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.314134 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.314215 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 20:59:44.314402 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.314972 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.316277 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.316731 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.316836 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.316846 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:44.317073 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 20:59:44.317143 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 20:59:44.317322 7333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfa2abf9939045fea80c1390a26d1d67" candidate_uuid: "a9aee071485248359bdc6f81ec939873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50d2b86d95c4d68832444bfe4a8b872" is_pre_election: true
I20260808 20:59:44.317512 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.313725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:42277 (local address 127.15.232.129:39207)
0808 20:59:44.313941 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:44.313951 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 20:59:44.314045 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 20:59:44.314219 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 20:59:44.314220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 20:59:44.314221 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 20:59:44.314310 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.314312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.315128 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.315129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.316139 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 20:59:44.316143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.316862 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.316864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 20:59:44.316934 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.316938 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 20:59:44.317330 (+ 392us) client_negotiation.cc:240] Negotiation successful
0808 20:59:44.317395 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":112,"threads_started":1}
I20260808 20:59:44.317543 7333 raft_consensus.cc:2454] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9aee071485248359bdc6f81ec939873 in term 0.
W20260808 20:59:44.317668 7408 tablet.cc:2369] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 20:59:44.317911 7043 leader_election.cc:302] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9aee071485248359bdc6f81ec939873, d50d2b86d95c4d68832444bfe4a8b872; no voters:
I20260808 20:59:44.317914 7535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfa2abf9939045fea80c1390a26d1d67" candidate_uuid: "a9aee071485248359bdc6f81ec939873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6328cca11e64cc5b610803f056c0da1" is_pre_election: true
I20260808 20:59:44.318101 7535 raft_consensus.cc:2454] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9aee071485248359bdc6f81ec939873 in term 0.
I20260808 20:59:44.318167 7216 raft_consensus.cc:2781] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 20:59:44.318264 7216 raft_consensus.cc:482] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 20:59:44.318295 7216 raft_consensus.cc:3037] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:44.319170 7216 raft_consensus.cc:504] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:44.319341 7216 leader_election.cc:288] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 1 election: Requested vote from peers d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853), a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277)
I20260808 20:59:44.319583 7333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfa2abf9939045fea80c1390a26d1d67" candidate_uuid: "a9aee071485248359bdc6f81ec939873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50d2b86d95c4d68832444bfe4a8b872"
I20260808 20:59:44.319576 7535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfa2abf9939045fea80c1390a26d1d67" candidate_uuid: "a9aee071485248359bdc6f81ec939873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6328cca11e64cc5b610803f056c0da1"
I20260808 20:59:44.319667 7535 raft_consensus.cc:3037] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:44.319667 7333 raft_consensus.cc:3037] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [term 0 FOLLOWER]: Advancing to term 1
I20260808 20:59:44.320591 7535 raft_consensus.cc:2454] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9aee071485248359bdc6f81ec939873 in term 1.
I20260808 20:59:44.320605 7333 raft_consensus.cc:2454] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9aee071485248359bdc6f81ec939873 in term 1.
I20260808 20:59:44.320811 7043 leader_election.cc:302] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9aee071485248359bdc6f81ec939873, d50d2b86d95c4d68832444bfe4a8b872; no voters:
I20260808 20:59:44.320926 7216 raft_consensus.cc:2781] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 20:59:44.321188 7216 raft_consensus.cc:686] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 LEADER]: Becoming Leader. State: Replica: a9aee071485248359bdc6f81ec939873, State: Running, Role: LEADER
I20260808 20:59:44.321295 7216 consensus_queue.cc:237] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 20:59:44.322333 6941 catalog_manager.cc:5495] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9aee071485248359bdc6f81ec939873 (127.15.232.129). New cstate: current_term: 1 leader_uuid: "a9aee071485248359bdc6f81ec939873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } health_report { overall_health: HEALTHY } } }
I20260808 20:59:44.342299 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.339565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.15.232.129:44565)
0808 20:59:44.339633 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 20:59:44.339635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 20:59:44.339644 (+ 9us) server_negotiation.cc:407] Connection header received
0808 20:59:44.339693 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 20:59:44.339694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 20:59:44.339715 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 20:59:44.339799 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 20:59:44.339919 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.340528 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.341424 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 20:59:44.341667 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 20:59:44.341753 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 20:59:44.341916 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 20:59:44.342044 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 20:59:44.342134 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 20:59:44.342180 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 20:59:44.342206 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 20:59:44.344252 7535 raft_consensus.cc:1261] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [term 1 FOLLOWER]: Refusing update from remote peer a9aee071485248359bdc6f81ec939873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:59:44.344331 7333 raft_consensus.cc:1261] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [term 1 FOLLOWER]: Refusing update from remote peer a9aee071485248359bdc6f81ec939873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 20:59:44.344713 7615 consensus_queue.cc:1035] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:59:44.344899 7216 consensus_queue.cc:1035] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 20:59:44.347378 7621 mvcc.cc:205] Tried to move back new op lower bound from 7316368524670992384 to 7316368524580814848. Current Snapshot: MvccSnapshot[applied={T|T < 7316368524670992384}]
I20260808 20:59:44.347973 7622 mvcc.cc:205] Tried to move back new op lower bound from 7316368524670992384 to 7316368524580814848. Current Snapshot: MvccSnapshot[applied={T|T < 7316368524670992384 or (T in {7316368524670992384})}]
I20260808 20:59:44.350152 6937 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:59:44.350266 6937 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 20:59:44.351161 6937 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Sending DeleteTablet for 3 replicas of tablet dfa2abf9939045fea80c1390a26d1d67
I20260808 20:59:44.351573 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 6995
I20260808 20:59:44.351580 7113 tablet_service.cc:1514] Processing DeleteTablet for tablet dfa2abf9939045fea80c1390a26d1d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:44 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:55622
I20260808 20:59:44.351562 7307 tablet_service.cc:1514] Processing DeleteTablet for tablet dfa2abf9939045fea80c1390a26d1d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:44 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:60134
I20260808 20:59:44.351677 7219 tablet_replica.cc:330] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: stopping tablet replica
I20260808 20:59:44.351737 7219 raft_consensus.cc:2229] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 LEADER]: Raft consensus shutting down.
I20260808 20:59:44.351821 7219 raft_consensus.cc:2258] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:44.357054 7505 tablet_service.cc:1514] Processing DeleteTablet for tablet dfa2abf9939045fea80c1390a26d1d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 20:59:44 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:40540
I20260808 20:59:44.357429 7636 tablet_replica.cc:330] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: stopping tablet replica
I20260808 20:59:44.357578 7636 raft_consensus.cc:2229] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:59:44.357723 7636 raft_consensus.cc:2258] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:44.358505 7636 ts_tablet_manager.cc:1905] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:44.360646 7636 ts_tablet_manager.cc:1918] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:44.360764 7636 log.cc:1196] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/dfa2abf9939045fea80c1390a26d1d67
I20260808 20:59:44.361187 7636 ts_tablet_manager.cc:1939] T dfa2abf9939045fea80c1390a26d1d67 P a6328cca11e64cc5b610803f056c0da1: Deleting consensus metadata
I20260808 20:59:44.362025 6925 catalog_manager.cc:4841] TS a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277): tablet dfa2abf9939045fea80c1390a26d1d67 (table test-table [id=16ce24b11ec244548c65297d4e498354]) successfully deleted
I20260808 20:59:44.362643 7634 tablet_replica.cc:330] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: stopping tablet replica
I20260808 20:59:44.362809 7634 raft_consensus.cc:2229] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 20:59:44.362980 7634 raft_consensus.cc:2258] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 20:59:44.363781 7634 ts_tablet_manager.cc:1905] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 20:59:44.365491 7634 ts_tablet_manager.cc:1918] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 20:59:44.365614 7634 log.cc:1196] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/dfa2abf9939045fea80c1390a26d1d67
I20260808 20:59:44.365935 7634 ts_tablet_manager.cc:1939] T dfa2abf9939045fea80c1390a26d1d67 P d50d2b86d95c4d68832444bfe4a8b872: Deleting consensus metadata
I20260808 20:59:44.366518 6924 catalog_manager.cc:4841] TS d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853): tablet dfa2abf9939045fea80c1390a26d1d67 (table test-table [id=16ce24b11ec244548c65297d4e498354]) successfully deleted
W20260808 20:59:44.374636 6924 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:44565 (error 108)
I20260808 20:59:44.375474 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 7215
I20260808 20:59:44.378285 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 20:59:44.378092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:44565 (local address 127.0.0.1:55632)
0808 20:59:44.378162 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 20:59:44.378233 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:44565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 20:59:44.380550 6924 catalog_manager.cc:4584] TS a9aee071485248359bdc6f81ec939873 (127.15.232.129:44565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dfa2abf9939045fea80c1390a26d1d67: Network error: Client connection negotiation failed: client connection to 127.15.232.129:44565: connect: Connection refused (error 111)
I20260808 20:59:44.388129 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 7410
I20260808 20:59:44.400974 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 6899
I20260808 21:00:04.418990 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35921
--webserver_interface=127.15.232.190
--webserver_port=45383
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:35921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:04.513019 7638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:04.513320 7638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:04.513365 7638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:04.514633 7638 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 21:00:04.514714 7638 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 21:00:04.514736 7638 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 21:00:04.514755 7638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:04.514784 7638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:04.514817 7638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:04.514842 7638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:04.514870 7638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:04.516472 7638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:35921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:35921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=45383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:04.516788 7638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:04.517037 7638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:04.523135 7644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:04.523303 7638 server_base.cc:1034] running on GCE node
W20260808 21:00:04.523113 7648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:04.523119 7645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:04.523761 7638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:04.524102 7638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:04.525269 7638 hybrid_clock.cc:648] HybridClock initialized: now 1786222804525222 us; error 64 us; skew 500 ppm
Aug 08 21:00:04 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:04.533169 7638 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:04.533757 7638 webserver.cc:460] Webserver started at http://127.15.232.190:45383/ using document root <none> and password file <none>
I20260808 21:00:04.534014 7638 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:04.534066 7638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:04.535634 7638 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:00:04.536687 7658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:04.537240 7638 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:04.537380 7638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "735e09c669ca46898c5455e3f6633e06"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "4058403eb6b1c2d9c5bc23a91927ce9b"
server_key_iv: "8928b354102537c3dfdb6e255bdd7005"
server_key_version: "encryptionkey@0"
I20260808 21:00:04.537945 7638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:04.571231 7638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:04.573992 7638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:04.574486 7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:04.579648 7638 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:35921
I20260808 21:00:04.579684 7731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:35921 every 8 connection(s)
I20260808 21:00:04.580109 7638 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:04.581753 7733 sys_catalog.cc:263] Verifying existing consensus state
I20260808 21:00:04.582540 7733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Bootstrap starting.
I20260808 21:00:04.587298 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 7638
I20260808 21:00:04.587747 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:44565
--local_ip_for_outbound_sockets=127.15.232.129
--tserver_master_addrs=127.15.232.190:35921
--webserver_port=34125
--webserver_interface=127.15.232.129
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 21:00:04.591922 7733 log.cc:824] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:04.594812 7733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 21:00:04.595204 7733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Bootstrap complete.
I20260808 21:00:04.597635 7733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 21:00:04.597906 7733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 735e09c669ca46898c5455e3f6633e06, State: Initialized, Role: FOLLOWER
I20260808 21:00:04.598132 7733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 21:00:04.598237 7733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:04.598295 7733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:04.598344 7733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 2 FOLLOWER]: Advancing to term 3
I20260808 21:00:04.599419 7733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 21:00:04.599591 7733 leader_election.cc:302] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 735e09c669ca46898c5455e3f6633e06; no voters:
I20260808 21:00:04.599833 7733 leader_election.cc:288] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 21:00:04.599875 7739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 21:00:04.599995 7739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [term 3 LEADER]: Becoming Leader. State: Replica: 735e09c669ca46898c5455e3f6633e06, State: Running, Role: LEADER
I20260808 21:00:04.600088 7739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } }
I20260808 21:00:04.600136 7733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:04.600371 7739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 735e09c669ca46898c5455e3f6633e06. Latest consensus state: current_term: 3 leader_uuid: "735e09c669ca46898c5455e3f6633e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } } }
I20260808 21:00:04.600449 7739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:04.600376 7741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "735e09c669ca46898c5455e3f6633e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "735e09c669ca46898c5455e3f6633e06" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 35921 } } }
I20260808 21:00:04.600629 7741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:04.600903 7745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:04.601797 7745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:04.601924 7745 catalog_manager.cc:1259] Loaded cluster ID: 0b4a91612c374ed991e2d5b2aaf0b74c
I20260808 21:00:04.601963 7745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:04.603219 7745 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:04.603417 7745 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Loaded TSK: 0
I20260808 21:00:04.603523 7745 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 735e09c669ca46898c5455e3f6633e06: Loaded TSK: 1
F20260808 21:00:04.603790 7745 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f59859c29ad google::LogMessage::Fail() at ??:0
@ 0x7f59859c588a google::LogMessage::SendToLog() at ??:0
@ 0x7f59859c24de google::LogMessage::Flush() at ??:0
@ 0x7f59859c3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f598812f104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f598812cd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f598d91c1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f598d91c48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f598d91c594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f598d8e2c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f598d8ecad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f59878900d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5987885954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f59886e96db start_thread at ??:0
@ 0x7f59844c061f clone at ??:0
W20260808 21:00:04.700567 7735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:04.700968 7735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:04.701011 7735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:04.702382 7735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:04.702437 7735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:04.702467 7735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 21:00:04.704200 7735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:44565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=34125
--tserver_master_addrs=127.15.232.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:04.704479 7735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:04.704747 7735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:04.710521 7800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:04.710644 7735 server_base.cc:1034] running on GCE node
W20260808 21:00:04.710479 7802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:04.710502 7798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:04.711027 7735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:04.711325 7735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:04.712528 7735 hybrid_clock.cc:648] HybridClock initialized: now 1786222804712499 us; error 41 us; skew 500 ppm
Aug 08 21:00:04 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:04.715937 7735 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 21:00:04.716562 7735 webserver.cc:460] Webserver started at http://127.15.232.129:34125/ using document root <none> and password file <none>
I20260808 21:00:04.716842 7735 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:04.716915 7735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:04.718766 7735 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:00:04.719704 7810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:04.719931 7735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:04.720049 7735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "a9aee071485248359bdc6f81ec939873"
format_stamp: "Formatted at 2026-08-08 20:59:22 on dist-test-slave-c12x"
server_key: "3b19b9798c0f49f2da86f1ed8fe8217d"
server_key_iv: "d9b6c798764360b437186f6c28221305"
server_key_version: "encryptionkey@0"
I20260808 21:00:04.720424 7735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:04.732436 7735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:04.735351 7735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:04.735598 7735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:04.735882 7735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:04.736382 7821 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 21:00:04.738777 7735 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 21:00:04.738870 7735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 21:00:04.738917 7735 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 21:00:04.739733 7735 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 21:00:04.739792 7735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:04.739892 7821 tablet_bootstrap.cc:492] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Bootstrap starting.
I20260808 21:00:04.747500 7735 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:44565
I20260808 21:00:04.747689 7963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:44565 every 8 connection(s)
I20260808 21:00:04.747983 7735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 21:00:04.751932 7821 log.cc:824] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:04.754782 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 7735
I20260808 21:00:04.754954 7821 tablet_bootstrap.cc:492] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 21:00:04.755170 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:41853
--local_ip_for_outbound_sockets=127.15.232.130
--tserver_master_addrs=127.15.232.190:35921
--webserver_port=38809
--webserver_interface=127.15.232.130
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 21:00:04.755326 7821 tablet_bootstrap.cc:492] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Bootstrap complete.
I20260808 21:00:04.756659 7821 ts_tablet_manager.cc:1397] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.013s
I20260808 21:00:04.758256 7821 raft_consensus.cc:348] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 21:00:04.758615 7821 raft_consensus.cc:729] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9aee071485248359bdc6f81ec939873, State: Initialized, Role: FOLLOWER
I20260808 21:00:04.758792 7821 consensus_queue.cc:260] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 21:00:04.759099 7821 ts_tablet_manager.cc:1428] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260808 21:00:04.825453 7967 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:00:04.748768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:44389)
0808 21:00:04.748959 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:04.749007 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:04.749523 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:04.825240 (+ 75717us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
W20260808 21:00:04.825759 7964 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:35921 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.190:35921: Client connection negotiation failed: client connection to 127.15.232.190:35921: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 21:00:04.826139 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:04.826005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:46337)
0808 21:00:04.826068 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:04.826114 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 21:00:04.826472 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:04.826381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:45061)
0808 21:00:04.826427 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:04.826454 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 21:00:04.826594 7964 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 21:00:04.851419 7970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:04.851743 7970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:04.851773 7970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:04.852970 7970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:04.853014 7970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:04.853042 7970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 21:00:04.854526 7970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:41853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=38809
--tserver_master_addrs=127.15.232.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:04.854729 7970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:04.854931 7970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:04.860809 7978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:04.860810 7981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:04.860984 7979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:04.861863 7970 server_base.cc:1034] running on GCE node
I20260808 21:00:04.862144 7970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:04.862443 7970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:04.863641 7970 hybrid_clock.cc:648] HybridClock initialized: now 1786222804863634 us; error 40 us; skew 500 ppm
Aug 08 21:00:04 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:04.866631 7970 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 21:00:04.867146 7970 webserver.cc:460] Webserver started at http://127.15.232.130:38809/ using document root <none> and password file <none>
I20260808 21:00:04.867379 7970 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:04.867460 7970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:04.868942 7970 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:04.869752 7988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:04.870105 7970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:04.870263 7970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "d50d2b86d95c4d68832444bfe4a8b872"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "b3a27b72170d2e63d2ec514cc76d7077"
server_key_iv: "65ac7882765552ab2c495cb79d1179c5"
server_key_version: "encryptionkey@0"
I20260808 21:00:04.870690 7970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:04.899591 7970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:04.902303 7970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:04.902539 7970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:04.902966 7970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:04.903349 7970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:04.903412 7970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:04.903472 7970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:04.903517 7970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:04.910519 7970 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:41853
I20260808 21:00:04.910565 8147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:41853 every 8 connection(s)
I20260808 21:00:04.910969 7970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 21:00:04.912231 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:04.911789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:35769)
0808 21:00:04.911989 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:04.912082 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
W20260808 21:00:04.912703 8149 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:35921 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.190:35921: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
I20260808 21:00:04.912981 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:04.912892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:60567)
0808 21:00:04.912936 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:04.912964 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 21:00:04.913257 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 7970
I20260808 21:00:04.913522 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:04.913395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:40597)
0808 21:00:04.913457 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:04.913485 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 21:00:04.913600 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:42277
--local_ip_for_outbound_sockets=127.15.232.131
--tserver_master_addrs=127.15.232.190:35921
--webserver_port=44671
--webserver_interface=127.15.232.131
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:04.913650 8149 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 21:00:05.001716 8152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:05.001938 8152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:05.001965 8152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:05.003095 8152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:05.003132 8152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:05.003158 8152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 21:00:05.005141 8152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:42277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=44671
--tserver_master_addrs=127.15.232.190:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:05.005360 8152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:05.005553 8152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:05.011096 8161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:05.011053 8158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:05.011255 8152 server_base.cc:1034] running on GCE node
W20260808 21:00:05.011078 8159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:05.011601 8152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:05.011859 8152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:05.013033 8152 hybrid_clock.cc:648] HybridClock initialized: now 1786222805013005 us; error 56 us; skew 500 ppm
Aug 08 21:00:05 dist-test-slave-c12x krb5kdc[6111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222805, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:05.017055 8152 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 21:00:05.017604 8152 webserver.cc:460] Webserver started at http://127.15.232.131:44671/ using document root <none> and password file <none>
I20260808 21:00:05.017858 8152 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:05.017907 8152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:05.019472 8152 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 21:00:05.020186 8168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:05.020372 8152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:05.020480 8152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "a6328cca11e64cc5b610803f056c0da1"
format_stamp: "Formatted at 2026-08-08 20:59:23 on dist-test-slave-c12x"
server_key: "835e9b5b07be4c6b3f8f3ca0f17d6539"
server_key_iv: "8d23665e1b0a438c763efdfd4ffcb256"
server_key_version: "encryptionkey@0"
I20260808 21:00:05.020895 8152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:05.037585 8152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:05.040819 8152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:05.041013 8152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:05.041281 8152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:05.041632 8152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:05.041689 8152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:05.041733 8152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:05.041766 8152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:05.047734 8152 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:42277
I20260808 21:00:05.047777 8333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:42277 every 8 connection(s)
I20260808 21:00:05.048132 8152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 21:00:05.049481 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:05.048810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:36847)
0808 21:00:05.049002 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:05.049080 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
W20260808 21:00:05.049882 8334 heartbeater.cc:644] Failed to heartbeat to 127.15.232.190:35921 (0 consecutive failures): Network error: Failed to ping master at 127.15.232.190:35921: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
I20260808 21:00:05.050201 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:05.050106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:38363)
0808 21:00:05.050152 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:05.050181 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 21:00:05.050544 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 8152
I20260808 21:00:05.050632 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:05.050512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:55191)
0808 21:00:05.050579 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:05.050606 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 21:00:05.050812 8334 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 21:00:05.827334 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:05.827063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:48589)
0808 21:00:05.827222 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:05.827266 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":77,"threads_started":1}
I20260808 21:00:05.914399 8342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:05.914102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:47895)
0808 21:00:05.914259 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:05.914313 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":63,"threads_started":1}
I20260808 21:00:06.051558 8343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:06.051180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:34835)
0808 21:00:06.051329 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:06.051367 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
I20260808 21:00:06.489463 8344 raft_consensus.cc:482] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:06.489602 8344 raft_consensus.cc:504] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 21:00:06.489930 8344 leader_election.cc:288] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853), a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277)
Aug 08 21:00:06 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.131@KRBTEST.COM
Aug 08 21:00:06 dist-test-slave-c12x krb5kdc[6111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222804, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.130@KRBTEST.COM
I20260808 21:00:06.505544 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:06.490159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:41853 (local address 127.15.232.129:54999)
0808 21:00:06.490623 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:06.490634 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:06.490728 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:06.491478 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:06.491481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:06.491932 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:06.492135 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:06.492142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:06.493387 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:06.493388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:06.494512 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:06.494515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:06.495917 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:06.495918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:06.495981 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:06.496965 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:06.496973 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:06.500070 (+ 3097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:06.503097 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:06.503098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:06.503099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:06.503420 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:06.503978 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:06.503979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:06.503979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:06.504174 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:06.504801 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:06.504813 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:06.505025 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 21:00:06.505163 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 21:00:06.505415 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":82,"thread_start_us":54,"threads_started":1}
I20260808 21:00:06.505638 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:06.490282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:54999 (local address 127.15.232.130:41853)
0808 21:00:06.490553 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:06.490566 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:06.490731 (+ 165us) server_negotiation.cc:407] Connection header received
0808 21:00:06.491249 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:06.491252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:06.491294 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:06.491474 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:06.492281 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:06.493238 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:06.494643 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:06.495720 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:06.495951 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:06.500255 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:06.500260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:06.500261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:06.500275 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:06.502988 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:06.503583 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:06.503590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:06.503590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:06.503852 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:06.504281 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:06.504282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:06.504282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:06.504656 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:06.504768 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:06.505340 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 21:00:06.505511 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":70,"threads_started":1}
I20260808 21:00:06.505638 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:06.490420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:38597 (local address 127.15.232.131:42277)
0808 21:00:06.490606 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:06.490615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:06.490650 (+ 35us) server_negotiation.cc:407] Connection header received
0808 21:00:06.491165 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:06.491167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:06.491212 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:06.491413 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:06.492286 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:06.493242 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:06.494636 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:06.495129 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:06.495413 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:06.499382 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:06.499387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:06.499388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:06.499401 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:06.502713 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:06.503601 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:06.503604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:06.503604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:06.503852 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:06.504281 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:06.504282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:06.504282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:06.504656 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:06.504768 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:06.505313 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 21:00:06.505511 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
I20260808 21:00:06.505667 8348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:06.490159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:42277 (local address 127.15.232.129:38597)
0808 21:00:06.490302 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:06.490315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:06.490408 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:06.491480 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:06.491482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:06.491935 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:06.492135 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:06.492142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:06.493402 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:06.493403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:06.494513 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:06.494516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:06.495380 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:06.495382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:06.495796 (+ 414us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:06.496584 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:06.496595 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:06.499171 (+ 2576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:06.502873 (+ 3702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:06.502882 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:06.502884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:06.503420 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:06.503979 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:06.503979 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:06.503980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:06.504174 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:06.504801 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:06.504813 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:06.505006 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 21:00:06.505169 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 21:00:06.505523 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":275,"thread_start_us":82,"threads_started":1}
W20260808 21:00:06.506935 7816 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
W20260808 21:00:06.507089 7815 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
I20260808 21:00:06.507154 7815 leader_election.cc:302] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9aee071485248359bdc6f81ec939873; no voters: a6328cca11e64cc5b610803f056c0da1, d50d2b86d95c4d68832444bfe4a8b872
I20260808 21:00:06.507270 8344 raft_consensus.cc:2726] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 21:00:06.828094 8348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:06.827911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:42861)
0808 21:00:06.828005 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:06.828050 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 21:00:06.915269 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:06.914972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:40039)
0808 21:00:06.915162 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:06.915217 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":91,"threads_started":1}
I20260808 21:00:07.052290 8353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:07.052061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:40393)
0808 21:00:07.052209 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:07.052245 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":66,"threads_started":1}
I20260808 21:00:07.828764 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:07.828530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:60475)
0808 21:00:07.828688 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:07.828719 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260808 21:00:07.916247 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:07.916013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:57555)
0808 21:00:07.916159 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:07.916201 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260808 21:00:08.053200 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:08.052903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:58049)
0808 21:00:08.053061 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:08.053135 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260808 21:00:08.085904 8359 raft_consensus.cc:482] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:08.086022 8359 raft_consensus.cc:504] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 21:00:08.086273 8359 leader_election.cc:288] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853), a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277)
W20260808 21:00:08.086887 7816 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
W20260808 21:00:08.086978 7815 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
I20260808 21:00:08.087018 7815 leader_election.cc:302] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9aee071485248359bdc6f81ec939873; no voters: a6328cca11e64cc5b610803f056c0da1, d50d2b86d95c4d68832444bfe4a8b872
I20260808 21:00:08.087126 8359 raft_consensus.cc:2726] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 21:00:08.829664 8360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:08.829379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:38405)
0808 21:00:08.829560 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:08.829611 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":80,"threads_started":1}
I20260808 21:00:08.917269 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:08.917013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:45967)
0808 21:00:08.917170 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:08.917217 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260808 21:00:09.054039 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:09.053779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:36247)
0808 21:00:09.053953 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:09.053992 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260808 21:00:09.830636 8363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:09.830331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:35205)
0808 21:00:09.830511 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:09.830590 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260808 21:00:09.918323 8365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:09.918048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:39387)
0808 21:00:09.918222 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:09.918272 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":78,"threads_started":1}
I20260808 21:00:10.054971 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:10.054672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:48779)
0808 21:00:10.054869 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:10.054918 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":88,"threads_started":1}
I20260808 21:00:10.193532 8367 raft_consensus.cc:482] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:10.193696 8367 raft_consensus.cc:504] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 21:00:10.193930 8367 leader_election.cc:288] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853), a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277)
W20260808 21:00:10.194460 7816 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
W20260808 21:00:10.194593 7815 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
I20260808 21:00:10.194659 7815 leader_election.cc:302] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9aee071485248359bdc6f81ec939873; no voters: a6328cca11e64cc5b610803f056c0da1, d50d2b86d95c4d68832444bfe4a8b872
I20260808 21:00:10.194890 8367 raft_consensus.cc:2726] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 21:00:10.831606 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:10.831248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:55643)
0808 21:00:10.831498 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:10.831557 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
I20260808 21:00:10.919332 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:10.919055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:41561)
0808 21:00:10.919229 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:10.919281 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":69,"threads_started":1}
I20260808 21:00:11.055989 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:11.055708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:53485)
0808 21:00:11.055866 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:11.055935 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
I20260808 21:00:11.832381 8373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:11.832148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:43943)
0808 21:00:11.832291 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:11.832329 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260808 21:00:11.920284 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:11.919992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:43115)
0808 21:00:11.920159 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:11.920214 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260808 21:00:12.056885 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:12.056646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:44399)
0808 21:00:12.056800 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:12.056837 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260808 21:00:12.833256 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:12.832993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:44237)
0808 21:00:12.833161 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:12.833208 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260808 21:00:12.860107 8378 raft_consensus.cc:482] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:12.860245 8378 raft_consensus.cc:504] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 21:00:12.860496 8378 leader_election.cc:288] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853), a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277)
W20260808 21:00:12.860953 7816 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
W20260808 21:00:12.861040 7815 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
I20260808 21:00:12.861148 7815 leader_election.cc:302] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9aee071485248359bdc6f81ec939873; no voters: a6328cca11e64cc5b610803f056c0da1, d50d2b86d95c4d68832444bfe4a8b872
I20260808 21:00:12.861263 8378 raft_consensus.cc:2726] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 21:00:12.921096 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:12.920825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:39211)
0808 21:00:12.920998 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:12.921048 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260808 21:00:13.057863 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:13.057503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:36841)
0808 21:00:13.057754 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:13.057812 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":72,"threads_started":1}
I20260808 21:00:13.834194 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:13.833880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:59937)
0808 21:00:13.834069 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:13.834129 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":105,"threads_started":1}
I20260808 21:00:13.921999 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:13.921765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:59225)
0808 21:00:13.921906 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:13.921946 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260808 21:00:14.058808 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:14.058511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:43419)
0808 21:00:14.058701 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:14.058749 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
I20260808 21:00:14.835198 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:14.834853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:53285)
0808 21:00:14.835038 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:14.835086 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260808 21:00:14.922983 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:14.922698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:52289)
0808 21:00:14.922884 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:14.922930 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260808 21:00:15.059717 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:15.059419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:36177)
0808 21:00:15.059624 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:15.059666 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":140,"threads_started":1}
I20260808 21:00:15.836337 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:15.835965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:46135)
0808 21:00:15.836189 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:15.836275 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
I20260808 21:00:15.923852 8396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:15.923585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:59289)
0808 21:00:15.923748 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:15.923803 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260808 21:00:16.060529 8398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:16.060267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:32913)
0808 21:00:16.060439 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:16.060480 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260808 21:00:16.837182 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:16.836942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:37823)
0808 21:00:16.837088 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:16.837134 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260808 21:00:16.924597 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:16.924319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:37207)
0808 21:00:16.924504 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:16.924543 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":87,"threads_started":1}
I20260808 21:00:17.061486 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:17.061128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:49785)
0808 21:00:17.061346 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:17.061398 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":99,"threads_started":1}
I20260808 21:00:17.838004 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:17.837716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:33111)
0808 21:00:17.837876 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:17.837919 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260808 21:00:17.925348 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:17.925116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:52713)
0808 21:00:17.925262 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:17.925301 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260808 21:00:18.062405 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:18.062115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:48339)
0808 21:00:18.062296 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:18.062346 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260808 21:00:18.839002 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:18.838727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:52211)
0808 21:00:18.838902 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:18.838952 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
I20260808 21:00:18.926277 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:18.926034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:55871)
0808 21:00:18.926178 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:18.926226 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260808 21:00:19.063372 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:19.063095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:51759)
0808 21:00:19.063266 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:19.063315 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260808 21:00:19.553040 8413 raft_consensus.cc:482] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:19.553149 8413 raft_consensus.cc:504] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50d2b86d95c4d68832444bfe4a8b872" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 41853 } } peers { permanent_uuid: "a6328cca11e64cc5b610803f056c0da1" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 42277 } } peers { permanent_uuid: "a9aee071485248359bdc6f81ec939873" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 44565 } }
I20260808 21:00:19.553351 8413 leader_election.cc:288] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853), a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277)
W20260808 21:00:19.553849 7815 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d50d2b86d95c4d68832444bfe4a8b872 (127.15.232.130:41853): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
W20260808 21:00:19.553925 7816 leader_election.cc:341] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6328cca11e64cc5b610803f056c0da1 (127.15.232.131:42277): Not found: Tablet not found: dfa2abf9939045fea80c1390a26d1d67
I20260808 21:00:19.553962 7816 leader_election.cc:302] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9aee071485248359bdc6f81ec939873; no voters: a6328cca11e64cc5b610803f056c0da1, d50d2b86d95c4d68832444bfe4a8b872
I20260808 21:00:19.554042 8413 raft_consensus.cc:2726] T dfa2abf9939045fea80c1390a26d1d67 P a9aee071485248359bdc6f81ec939873 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 21:00:19.840010 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:19.839755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.129:36437)
0808 21:00:19.839919 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:19.839965 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260808 21:00:19.927284 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:19.927003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.130:38167)
0808 21:00:19.927192 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:19.927239 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260808 21:00:20.057400 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 7735
I20260808 21:00:20.065459 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:20.065219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:35921 (local address 127.15.232.131:46021)
0808 21:00:20.065361 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:20.065408 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.190:35921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260808 21:00:20.071208 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 7970
I20260808 21:00:20.083925 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 8152
2026-08-08T21:00:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58790 ms)
[----------] 30 tests from SecurityITest (246522 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:20 dist-test-slave-c12x krb5kdc[8424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:20 dist-test-slave-c12x krb5kdc[8424](info): set up 2 sockets
Aug 08 21:00:20 dist-test-slave-c12x krb5kdc[8424](info): commencing operation
krb5kdc: starting...
W20260808 21:00:21.374189 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:21 dist-test-slave-c12x krb5kdc[8424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:00:21.512754 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33113
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:35179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:33113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:21.602744 8443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:21.603055 8443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:21.603103 8443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:21.604384 8443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:21.604466 8443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:21.604491 8443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:21.604526 8443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:21.604553 8443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:21.606096 8443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:35179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:33113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:21.606400 8443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:21.606647 8443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:21.612061 8448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:21.612134 8449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:21.612260 8443 server_base.cc:1034] running on GCE node
W20260808 21:00:21.612067 8451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:21.612668 8443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:21.613036 8443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:21.614248 8443 hybrid_clock.cc:648] HybridClock initialized: now 1786222821614232 us; error 34 us; skew 500 ppm
Aug 08 21:00:21 dist-test-slave-c12x krb5kdc[8424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222821, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:21.617447 8443 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:21.618170 8443 webserver.cc:460] Webserver started at http://127.15.232.190:39585/ using document root <none> and password file <none>
I20260808 21:00:21.618484 8443 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:21.618568 8443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:21.618774 8443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:21.620126 8443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "5002307d0b5d4495a398063172adaa2d"
format_stamp: "Formatted at 2026-08-08 21:00:21 on dist-test-slave-c12x"
server_key: "068263c250c9f703ec46497af2fe50e2"
server_key_iv: "255d711fb604c11b4c7476ea09c16f24"
server_key_version: "encryptionkey@0"
I20260808 21:00:21.620639 8443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "5002307d0b5d4495a398063172adaa2d"
format_stamp: "Formatted at 2026-08-08 21:00:21 on dist-test-slave-c12x"
server_key: "068263c250c9f703ec46497af2fe50e2"
server_key_iv: "255d711fb604c11b4c7476ea09c16f24"
server_key_version: "encryptionkey@0"
I20260808 21:00:21.622768 8443 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 21:00:21.623929 8460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:21.624143 8443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 21:00:21.624251 8443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "5002307d0b5d4495a398063172adaa2d"
format_stamp: "Formatted at 2026-08-08 21:00:21 on dist-test-slave-c12x"
server_key: "068263c250c9f703ec46497af2fe50e2"
server_key_iv: "255d711fb604c11b4c7476ea09c16f24"
server_key_version: "encryptionkey@0"
I20260808 21:00:21.624337 8443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:21.632849 8443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:21.635272 8443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:21.635506 8443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:21.640192 8443 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:33113
I20260808 21:00:21.640268 8536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:33113 every 8 connection(s)
I20260808 21:00:21.640637 8443 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:21.641607 8537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:21.644192 8537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d: Bootstrap starting.
I20260808 21:00:21.644869 8537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:21.645188 8537 log.cc:824] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:21.646176 8537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d: No bootstrap required, opened a new log
I20260808 21:00:21.648010 8537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5002307d0b5d4495a398063172adaa2d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33113 } }
I20260808 21:00:21.648169 8537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:21.648192 8537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5002307d0b5d4495a398063172adaa2d, State: Initialized, Role: FOLLOWER
I20260808 21:00:21.648379 8537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5002307d0b5d4495a398063172adaa2d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33113 } }
I20260808 21:00:21.648456 8537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:21.648479 8537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:21.648531 8537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:21.649169 8537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5002307d0b5d4495a398063172adaa2d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33113 } }
I20260808 21:00:21.649292 8537 leader_election.cc:302] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5002307d0b5d4495a398063172adaa2d; no voters:
I20260808 21:00:21.649523 8537 leader_election.cc:288] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:21.649581 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 8443
I20260808 21:00:21.649564 8540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:21.649731 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:21.649747 8540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [term 1 LEADER]: Becoming Leader. State: Replica: 5002307d0b5d4495a398063172adaa2d, State: Running, Role: LEADER
I20260808 21:00:21.649845 8537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:21.649842 8540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5002307d0b5d4495a398063172adaa2d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33113 } }
I20260808 21:00:21.649966 16290 external_mini_cluster.cc:1428] Setting key 2ca849e87ae3dd29c66c6350d8d47ac8
I20260808 21:00:21.650204 8541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5002307d0b5d4495a398063172adaa2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5002307d0b5d4495a398063172adaa2d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33113 } } }
I20260808 21:00:21.650300 8541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:21.650568 8542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5002307d0b5d4495a398063172adaa2d. Latest consensus state: current_term: 1 leader_uuid: "5002307d0b5d4495a398063172adaa2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5002307d0b5d4495a398063172adaa2d" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33113 } } }
I20260808 21:00:21.650696 8542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:21.650955 8544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:21.651609 8544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:21.653502 8544 catalog_manager.cc:1347] Generated new cluster ID: 7453b2755d084e6a9405b9e019eaf504
I20260808 21:00:21.653556 8544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 21:00:21 dist-test-slave-c12x krb5kdc[8424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:21.661249 8544 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:21.661943 8544 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:21.663167 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:21.650693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.15.232.190:33113)
0808 21:00:21.650861 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:21.650873 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:21.651216 (+ 343us) server_negotiation.cc:407] Connection header received
0808 21:00:21.651354 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:21.651362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:21.651460 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:21.651597 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:21.652679 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:21.653924 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:21.655060 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:21.655716 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:21.655820 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:21.658761 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:21.658766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:21.658767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:21.658780 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:21.661122 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:21.661615 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:21.661619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:21.661619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:21.661751 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:21.662054 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:21.662055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:21.662056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:21.662426 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:21.662651 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:21.662730 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 21:00:21.662908 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260808 21:00:21.665915 8544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d: Generated new TSK 0
I20260808 21:00:21.666136 8544 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 21:00:21.698230 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33113
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:35179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:21.786772 8573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:21.787035 8573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:21.787089 8573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:21.788393 8573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:21.788467 8573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:21.788513 8573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 21:00:21.790089 8573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:35179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:21.790366 8573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:21.790618 8573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:21.797220 8582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:21.797255 8584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:21.797391 8587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:21.797791 8573 server_base.cc:1034] running on GCE node
I20260808 21:00:21.798014 8573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:21.798285 8573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:21.799491 8573 hybrid_clock.cc:648] HybridClock initialized: now 1786222821799475 us; error 28 us; skew 500 ppm
Aug 08 21:00:21 dist-test-slave-c12x krb5kdc[8424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222821, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:21.802739 8573 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 21:00:21.803282 8573 webserver.cc:460] Webserver started at http://127.15.232.129:36727/ using document root <none> and password file <none>
I20260808 21:00:21.803586 8573 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:21.803639 8573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:21.803776 8573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:21.804960 8573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "10cd42aa62634b96a9ee57abfe0a2d32"
format_stamp: "Formatted at 2026-08-08 21:00:21 on dist-test-slave-c12x"
server_key: "8230082db534667107c80d0d50cecdd1"
server_key_iv: "27f91cd639e73614685918ff22a852bc"
server_key_version: "encryptionkey@0"
I20260808 21:00:21.805339 8573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "10cd42aa62634b96a9ee57abfe0a2d32"
format_stamp: "Formatted at 2026-08-08 21:00:21 on dist-test-slave-c12x"
server_key: "8230082db534667107c80d0d50cecdd1"
server_key_iv: "27f91cd639e73614685918ff22a852bc"
server_key_version: "encryptionkey@0"
I20260808 21:00:21.806929 8573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 21:00:21.807794 8598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:21.808342 8573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:21.808435 8573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "10cd42aa62634b96a9ee57abfe0a2d32"
format_stamp: "Formatted at 2026-08-08 21:00:21 on dist-test-slave-c12x"
server_key: "8230082db534667107c80d0d50cecdd1"
server_key_iv: "27f91cd639e73614685918ff22a852bc"
server_key_version: "encryptionkey@0"
I20260808 21:00:21.808573 8573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:21.822464 8573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:21.825289 8573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:21.825464 8573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:21.825745 8573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:21.826102 8573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:21.826164 8573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:21.826212 8573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:21.826241 8573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:21.833213 8573 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:42931
I20260808 21:00:21.833264 8763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:42931 every 8 connection(s)
I20260808 21:00:21.833642 8573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 21:00:21.835121 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 8573
I20260808 21:00:21.835265 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 21:00:21.835547 16290 external_mini_cluster.cc:1428] Setting key a81a22079f1e4c5b2de227277ae4e7fb
Aug 08 21:00:21 dist-test-slave-c12x krb5kdc[8424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222821, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
I20260808 21:00:21.845271 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:21.834569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:38677 (local address 127.15.232.190:33113)
0808 21:00:21.834652 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:21.834655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:21.834893 (+ 238us) server_negotiation.cc:407] Connection header received
0808 21:00:21.835583 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:21.835584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:21.835598 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:21.835660 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:21.837012 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:21.837583 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:21.838715 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:21.838957 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:21.839060 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:21.841827 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:21.841830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:21.841830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:21.841839 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:21.843419 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:21.844090 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:21.844091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:21.844092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:21.844169 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:21.844494 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:21.844496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:21.844496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:21.844633 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:21.844718 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:21.845058 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 21:00:21.845167 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 21:00:21.845448 8767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:21.834527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33113 (local address 127.15.232.129:38677)
0808 21:00:21.834745 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:21.834791 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:21.835458 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:21.835739 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:21.835743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:21.836669 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:21.836886 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:21.836892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:21.837696 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:21.837698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:21.838608 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:21.838610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:21.839073 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:21.839075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:21.839223 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:21.839904 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:21.839914 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:21.841709 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:21.843623 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:21.843633 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:21.843636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:21.843961 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:21.844251 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:21.844252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:21.844252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:21.844388 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:21.844708 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:21.844716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:21.844866 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 21:00:21.844994 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 21:00:21.845175 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":116,"threads_started":1}
I20260808 21:00:21.846105 8764 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33113
I20260808 21:00:21.846212 8764 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:21.846367 8764 heartbeater.cc:511] Master 127.15.232.190:33113 requested a full tablet report, sending...
I20260808 21:00:21.847064 8482 ts_manager.cc:194] Registered new tserver with Master: 10cd42aa62634b96a9ee57abfe0a2d32 (127.15.232.129:42931)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
I20260808 21:00:21.847980 8482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:38677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 21:00:21.864658 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33113
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:35179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:21.950412 8770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:21.950692 8770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:21.950736 8770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:21.952183 8770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:21.952260 8770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:21.952316 8770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 21:00:21.953951 8770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:35179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:21.954257 8770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:21.954520 8770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:21.960378 8775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:21.960513 8781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:21.960378 8776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:21.960757 8770 server_base.cc:1034] running on GCE node
I20260808 21:00:21.960969 8770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:21.961292 8770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:21.962495 8770 hybrid_clock.cc:648] HybridClock initialized: now 1786222821962453 us; error 63 us; skew 500 ppm
Aug 08 21:00:21 dist-test-slave-c12x krb5kdc[8424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222821, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:21.965530 8770 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 21:00:21.966138 8770 webserver.cc:460] Webserver started at http://127.15.232.130:39221/ using document root <none> and password file <none>
I20260808 21:00:21.966451 8770 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:21.966532 8770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:21.966709 8770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:21.968037 8770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "1618cc69088941b1bcf3c47c452e69b9"
format_stamp: "Formatted at 2026-08-08 21:00:21 on dist-test-slave-c12x"
server_key: "2ecdaa73784c06599a7eb249a896fb7c"
server_key_iv: "18b69884cc3d55e689038925a9d7248e"
server_key_version: "encryptionkey@0"
I20260808 21:00:21.968472 8770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "1618cc69088941b1bcf3c47c452e69b9"
format_stamp: "Formatted at 2026-08-08 21:00:21 on dist-test-slave-c12x"
server_key: "2ecdaa73784c06599a7eb249a896fb7c"
server_key_iv: "18b69884cc3d55e689038925a9d7248e"
server_key_version: "encryptionkey@0"
I20260808 21:00:21.970255 8770 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:21.971381 8796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:21.972029 8770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:21.972172 8770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "1618cc69088941b1bcf3c47c452e69b9"
format_stamp: "Formatted at 2026-08-08 21:00:21 on dist-test-slave-c12x"
server_key: "2ecdaa73784c06599a7eb249a896fb7c"
server_key_iv: "18b69884cc3d55e689038925a9d7248e"
server_key_version: "encryptionkey@0"
I20260808 21:00:21.972263 8770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:21.986351 8770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:21.988945 8770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:21.989146 8770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:21.989413 8770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:21.989765 8770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:21.989825 8770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:21.989868 8770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:21.989897 8770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:21.998091 8770 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:38221
I20260808 21:00:21.998155 8950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:38221 every 8 connection(s)
I20260808 21:00:21.998541 8770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 21:00:22.001418 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 8770
I20260808 21:00:22.001554 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 21:00:22.001770 16290 external_mini_cluster.cc:1428] Setting key 04e7805952662c73b054986382bcd156
Aug 08 21:00:21 dist-test-slave-c12x krb5kdc[8424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222821, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:22.009794 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:21.999492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:59131 (local address 127.15.232.190:33113)
0808 21:00:21.999540 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:21.999543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:21.999772 (+ 229us) server_negotiation.cc:407] Connection header received
0808 21:00:22.000285 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.000286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.000300 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.000369 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:22.001464 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.002220 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.003407 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.003803 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.003889 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.006384 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:22.006385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:22.006386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:22.006393 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:22.007979 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:22.008572 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:22.008573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:22.008573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:22.008643 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:22.008975 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:22.008976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:22.008976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:22.009091 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:22.009156 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.009596 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.009668 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 21:00:22.010082 8953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:21.999483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33113 (local address 127.15.232.130:59131)
0808 21:00:21.999656 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:21.999694 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:22.000176 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:22.000425 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:22.000428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:22.001031 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:22.001312 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.001321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.002323 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.002324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.003292 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.003296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.003897 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.003898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.004008 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.004608 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:22.004617 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:22.006276 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:22.008075 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:22.008085 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:22.008088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:22.008471 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:22.008722 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:22.008723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:22.008724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:22.008901 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:22.009179 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:22.009187 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:22.009402 (+ 215us) client_negotiation.cc:769] Sending connection context
0808 21:00:22.009556 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 21:00:22.009785 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260808 21:00:22.010728 8951 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33113
I20260808 21:00:22.010869 8951 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:22.011102 8951 heartbeater.cc:511] Master 127.15.232.190:33113 requested a full tablet report, sending...
I20260808 21:00:22.011570 8482 ts_manager.cc:194] Registered new tserver with Master: 1618cc69088941b1bcf3c47c452e69b9 (127.15.232.130:38221)
I20260808 21:00:22.012217 8482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:59131
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 21:00:22.032573 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33113
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:35179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:22.124197 8957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:22.124408 8957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:22.124432 8957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:22.125524 8957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:22.125561 8957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:22.125594 8957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 21:00:22.127004 8957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:35179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:33113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:22.127190 8957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:22.127372 8957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:22.132865 8969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:22.132948 8965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:22.132896 8964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:22.133121 8957 server_base.cc:1034] running on GCE node
I20260808 21:00:22.133435 8957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:22.133670 8957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:22.134867 8957 hybrid_clock.cc:648] HybridClock initialized: now 1786222822134850 us; error 52 us; skew 500 ppm
Aug 08 21:00:22 dist-test-slave-c12x krb5kdc[8424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222822, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:22.137686 8957 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 21:00:22.138155 8957 webserver.cc:460] Webserver started at http://127.15.232.131:32955/ using document root <none> and password file <none>
I20260808 21:00:22.138393 8957 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:22.138463 8957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:22.138652 8957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:22.139938 8957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "316c3a7af34046ad83c9078829ebe2af"
format_stamp: "Formatted at 2026-08-08 21:00:22 on dist-test-slave-c12x"
server_key: "73adaf8438d52c0eca7b0987f926e067"
server_key_iv: "e9f9ce390185058950aac222dc317c97"
server_key_version: "encryptionkey@0"
I20260808 21:00:22.140328 8957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "316c3a7af34046ad83c9078829ebe2af"
format_stamp: "Formatted at 2026-08-08 21:00:22 on dist-test-slave-c12x"
server_key: "73adaf8438d52c0eca7b0987f926e067"
server_key_iv: "e9f9ce390185058950aac222dc317c97"
server_key_version: "encryptionkey@0"
I20260808 21:00:22.141783 8957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:00:22.142602 8980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:22.142834 8957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:22.142949 8957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "316c3a7af34046ad83c9078829ebe2af"
format_stamp: "Formatted at 2026-08-08 21:00:22 on dist-test-slave-c12x"
server_key: "73adaf8438d52c0eca7b0987f926e067"
server_key_iv: "e9f9ce390185058950aac222dc317c97"
server_key_version: "encryptionkey@0"
I20260808 21:00:22.143030 8957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:22.161554 8957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:22.164252 8957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:22.164423 8957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:22.164687 8957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:22.165020 8957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:22.165077 8957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:22.165114 8957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:22.165158 8957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:22.171175 8957 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:39717
I20260808 21:00:22.171265 9131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:39717 every 8 connection(s)
I20260808 21:00:22.171664 8957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
Aug 08 21:00:22 dist-test-slave-c12x krb5kdc[8424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222822, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:22.179414 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 8957
I20260808 21:00:22.179807 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 21:00:22.180009 16290 external_mini_cluster.cc:1428] Setting key 598785ae12ff0624e05123add30cca4d
I20260808 21:00:22.181996 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.172488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:59349 (local address 127.15.232.190:33113)
0808 21:00:22.172562 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:22.172564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:22.172777 (+ 213us) server_negotiation.cc:407] Connection header received
0808 21:00:22.173251 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.173252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.173263 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.173341 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:22.174285 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.174868 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.176102 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.176308 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.176374 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.178679 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:22.178681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:22.178681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:22.178697 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:22.180694 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:22.181152 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:22.181153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:22.181153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:22.181208 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:22.181428 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:22.181429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:22.181429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:22.181552 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:22.181619 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.181857 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.181930 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 21:00:22.182163 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.172407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:33113 (local address 127.15.232.131:59349)
0808 21:00:22.172615 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:22.172660 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:22.173138 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:22.173398 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:22.173401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:22.173976 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:22.174176 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.174182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.174977 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.174978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.176010 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.176013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.176390 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.176391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.176521 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.177095 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:22.177104 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:22.178563 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:22.180758 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:22.180765 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:22.180767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:22.181081 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:22.181256 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:22.181256 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:22.181257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:22.181347 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:22.181621 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:22.181626 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:22.181741 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 21:00:22.181825 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 21:00:22.181958 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260808 21:00:22.182605 9134 heartbeater.cc:347] Connected to a master server at 127.15.232.190:33113
I20260808 21:00:22.182729 9134 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:22.182910 9134 heartbeater.cc:511] Master 127.15.232.190:33113 requested a full tablet report, sending...
I20260808 21:00:22.183351 8482 ts_manager.cc:194] Registered new tserver with Master: 316c3a7af34046ad83c9078829ebe2af (127.15.232.131:39717)
I20260808 21:00:22.183782 8482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:59349
I20260808 21:00:22.192237 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 21:00:22.200189 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.193658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.15.232.190:33113)
0808 21:00:22.193733 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:22.193735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:22.193877 (+ 142us) server_negotiation.cc:407] Connection header received
0808 21:00:22.193934 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.193935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.193944 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.194005 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:22.194609 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.195095 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.196031 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.196216 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.196272 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.197551 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:22.197552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:22.197552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:22.197559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:22.198992 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:22.199400 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:22.199401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:22.199401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:22.199512 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:22.199723 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:22.199724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:22.199724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:22.199881 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:22.199961 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.200053 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.200120 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 21:00:22.202013 8482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 21:00:22.203783 8482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 21:00:22.210958 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.206306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:42931 (local address 127.0.0.1:41596)
0808 21:00:22.206633 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:22.206674 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:22.206781 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:22.207072 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:22.207075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:22.207077 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:22.207163 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.207165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.208651 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.208652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.209747 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.209750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.210579 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.210581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.210697 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.210715 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 21:00:22.210801 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 21:00:22.210859 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":45,"thread_start_us":95,"threads_started":1}
I20260808 21:00:22.210989 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.206440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41596 (local address 127.15.232.129:42931)
0808 21:00:22.206620 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:22.206625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:22.206759 (+ 134us) server_negotiation.cc:407] Connection header received
0808 21:00:22.206939 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.206942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.206960 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.207251 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:22.207337 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.208510 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.209841 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.210491 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.210566 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.210623 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.210832 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.210877 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260808 21:00:22.211397 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.206560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:38221 (local address 127.0.0.1:57450)
0808 21:00:22.207000 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:22.207011 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:22.207073 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:22.207357 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:22.207358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:22.207361 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:22.207401 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.207403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.208699 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.208700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.210126 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.210130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.211077 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.211078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.211154 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.211159 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 21:00:22.211244 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 21:00:22.211290 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":31,"thread_start_us":80,"threads_started":1}
I20260808 21:00:22.211575 9153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.206349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:39717 (local address 127.0.0.1:42754)
0808 21:00:22.206707 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:22.206721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:22.206782 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:22.207388 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:22.207389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:22.207392 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:22.207471 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.207473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.208419 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.208420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.210144 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.210146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.211257 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.211258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.211334 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.211338 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:22.211399 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 21:00:22.211496 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":102,"thread_start_us":108,"threads_started":1}
I20260808 21:00:22.211828 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.206812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.15.232.130:38221)
0808 21:00:22.206999 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:22.207003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:22.207141 (+ 138us) server_negotiation.cc:407] Connection header received
0808 21:00:22.207185 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.207187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.207203 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.207278 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:22.207516 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.208600 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.210229 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.210911 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.210984 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.211033 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.211606 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.211731 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260808 21:00:22.211829 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.206472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42754 (local address 127.15.232.131:39717)
0808 21:00:22.207213 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:22.207218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:22.207229 (+ 11us) server_negotiation.cc:407] Connection header received
0808 21:00:22.207271 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.207273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.207290 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.207377 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:22.207537 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.208302 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.210247 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.211104 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.211217 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.211279 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.211693 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.211740 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":131,"threads_started":1}
I20260808 21:00:22.212564 8672 tablet_service.cc:1467] Processing CreateTablet for tablet 188883482ab147cc83ce3933ba8b5c82 (DEFAULT_TABLE table=test-table [id=162cff717f17415993a3f0ec1e88907f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:22.212724 9044 tablet_service.cc:1467] Processing CreateTablet for tablet 188883482ab147cc83ce3933ba8b5c82 (DEFAULT_TABLE table=test-table [id=162cff717f17415993a3f0ec1e88907f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:22.213035 8672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 188883482ab147cc83ce3933ba8b5c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:22.213042 9044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 188883482ab147cc83ce3933ba8b5c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:22.216957 9160 tablet_bootstrap.cc:492] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32: Bootstrap starting.
I20260808 21:00:22.217384 8866 tablet_service.cc:1467] Processing CreateTablet for tablet 188883482ab147cc83ce3933ba8b5c82 (DEFAULT_TABLE table=test-table [id=162cff717f17415993a3f0ec1e88907f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:22.217723 9160 tablet_bootstrap.cc:654] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:22.217760 8866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 188883482ab147cc83ce3933ba8b5c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:22.218351 9160 log.cc:824] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:22.218772 9159 tablet_bootstrap.cc:492] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: Bootstrap starting.
I20260808 21:00:22.219355 9159 tablet_bootstrap.cc:654] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:22.219844 9159 log.cc:824] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:22.220489 9160 tablet_bootstrap.cc:492] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32: No bootstrap required, opened a new log
I20260808 21:00:22.220574 9160 ts_tablet_manager.cc:1397] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 21:00:22.220793 9159 tablet_bootstrap.cc:492] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: No bootstrap required, opened a new log
I20260808 21:00:22.220881 9159 ts_tablet_manager.cc:1397] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:22.222223 9160 raft_consensus.cc:348] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } }
I20260808 21:00:22.222452 9160 raft_consensus.cc:374] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:22.222498 9160 raft_consensus.cc:729] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10cd42aa62634b96a9ee57abfe0a2d32, State: Initialized, Role: FOLLOWER
I20260808 21:00:22.222625 9160 consensus_queue.cc:260] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } }
I20260808 21:00:22.222755 9159 raft_consensus.cc:348] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } }
I20260808 21:00:22.222898 9159 raft_consensus.cc:374] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:22.222929 9159 raft_consensus.cc:729] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 316c3a7af34046ad83c9078829ebe2af, State: Initialized, Role: FOLLOWER
I20260808 21:00:22.222918 9160 ts_tablet_manager.cc:1428] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 21:00:22.222967 8764 heartbeater.cc:503] Master 127.15.232.190:33113 was elected leader, sending a full tablet report...
I20260808 21:00:22.223060 9159 consensus_queue.cc:260] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } }
I20260808 21:00:22.223136 9163 tablet_bootstrap.cc:492] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: Bootstrap starting.
I20260808 21:00:22.223363 9159 ts_tablet_manager.cc:1428] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:22.223508 9134 heartbeater.cc:503] Master 127.15.232.190:33113 was elected leader, sending a full tablet report...
I20260808 21:00:22.223847 9163 tablet_bootstrap.cc:654] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:22.224519 9163 log.cc:824] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:22.225405 9163 tablet_bootstrap.cc:492] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: No bootstrap required, opened a new log
I20260808 21:00:22.225459 9163 ts_tablet_manager.cc:1397] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 21:00:22.226787 9163 raft_consensus.cc:348] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } }
I20260808 21:00:22.226923 9163 raft_consensus.cc:374] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:22.226946 9163 raft_consensus.cc:729] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1618cc69088941b1bcf3c47c452e69b9, State: Initialized, Role: FOLLOWER
I20260808 21:00:22.227061 9163 consensus_queue.cc:260] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } }
I20260808 21:00:22.227355 9163 ts_tablet_manager.cc:1428] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 21:00:22.227458 8951 heartbeater.cc:503] Master 127.15.232.190:33113 was elected leader, sending a full tablet report...
W20260808 21:00:22.249194 8952 tablet.cc:2369] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 21:00:22.334735 8765 tablet.cc:2369] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 21:00:22.422308 9135 tablet.cc:2369] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 21:00:22.610620 9166 raft_consensus.cc:482] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:22.610759 9166 raft_consensus.cc:504] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } }
I20260808 21:00:22.611076 9166 leader_election.cc:288] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10cd42aa62634b96a9ee57abfe0a2d32 (127.15.232.129:42931), 1618cc69088941b1bcf3c47c452e69b9 (127.15.232.130:38221)
I20260808 21:00:22.615563 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.611527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:58343 (local address 127.15.232.129:42931)
0808 21:00:22.611602 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:22.611604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:22.611613 (+ 9us) server_negotiation.cc:407] Connection header received
0808 21:00:22.611643 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.611645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.611660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.611718 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:22.611912 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.612718 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.614578 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.615117 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.615199 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.615208 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.615419 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.615489 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 21:00:22.615563 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.611611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:54209 (local address 127.15.232.130:38221)
0808 21:00:22.611672 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:22.611674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:22.611805 (+ 131us) server_negotiation.cc:407] Connection header received
0808 21:00:22.611854 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.611855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.611864 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.611939 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:22.612093 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.612764 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.614558 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.615131 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.615199 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.615208 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.615419 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.615480 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 21:00:22.615584 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.611312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:42931 (local address 127.15.232.131:58343)
0808 21:00:22.611375 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:22.611388 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:22.611483 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:22.611752 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:22.611753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:22.611755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:22.611835 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.611837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.612866 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.612867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.614446 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.614450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.615224 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.615225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.615331 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.615333 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 21:00:22.615404 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 21:00:22.615487 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":107}
I20260808 21:00:22.615593 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.611494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:38221 (local address 127.15.232.131:54209)
0808 21:00:22.611678 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:22.611691 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:22.611795 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:22.611963 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:22.611964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:22.611965 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:22.612014 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.612015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.612866 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.612867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.614446 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:22.614450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.615224 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.615225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:22.615328 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.615333 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 21:00:22.615404 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 21:00:22.615502 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":89,"thread_start_us":107,"threads_started":1}
I20260808 21:00:22.616041 8891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188883482ab147cc83ce3933ba8b5c82" candidate_uuid: "316c3a7af34046ad83c9078829ebe2af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1618cc69088941b1bcf3c47c452e69b9" is_pre_election: true
I20260808 21:00:22.616030 8697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188883482ab147cc83ce3933ba8b5c82" candidate_uuid: "316c3a7af34046ad83c9078829ebe2af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" is_pre_election: true
I20260808 21:00:22.616220 8891 raft_consensus.cc:2454] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 316c3a7af34046ad83c9078829ebe2af in term 0.
I20260808 21:00:22.616220 8697 raft_consensus.cc:2454] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 316c3a7af34046ad83c9078829ebe2af in term 0.
I20260808 21:00:22.616495 8981 leader_election.cc:302] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1618cc69088941b1bcf3c47c452e69b9, 316c3a7af34046ad83c9078829ebe2af; no voters:
I20260808 21:00:22.616631 9166 raft_consensus.cc:2781] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 21:00:22.616691 9166 raft_consensus.cc:482] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 21:00:22.616716 9166 raft_consensus.cc:3037] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:22.617435 9166 raft_consensus.cc:504] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } }
I20260808 21:00:22.617579 9166 leader_election.cc:288] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [CANDIDATE]: Term 1 election: Requested vote from peers 10cd42aa62634b96a9ee57abfe0a2d32 (127.15.232.129:42931), 1618cc69088941b1bcf3c47c452e69b9 (127.15.232.130:38221)
I20260808 21:00:22.617803 8697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188883482ab147cc83ce3933ba8b5c82" candidate_uuid: "316c3a7af34046ad83c9078829ebe2af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10cd42aa62634b96a9ee57abfe0a2d32"
I20260808 21:00:22.617822 8891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "188883482ab147cc83ce3933ba8b5c82" candidate_uuid: "316c3a7af34046ad83c9078829ebe2af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1618cc69088941b1bcf3c47c452e69b9"
I20260808 21:00:22.617925 8891 raft_consensus.cc:3037] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:22.617914 8697 raft_consensus.cc:3037] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:22.618887 8697 raft_consensus.cc:2454] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 316c3a7af34046ad83c9078829ebe2af in term 1.
I20260808 21:00:22.618894 8891 raft_consensus.cc:2454] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 316c3a7af34046ad83c9078829ebe2af in term 1.
I20260808 21:00:22.619117 8981 leader_election.cc:302] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10cd42aa62634b96a9ee57abfe0a2d32, 316c3a7af34046ad83c9078829ebe2af; no voters:
I20260808 21:00:22.619226 9166 raft_consensus.cc:2781] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:22.619382 9166 raft_consensus.cc:686] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 1 LEADER]: Becoming Leader. State: Replica: 316c3a7af34046ad83c9078829ebe2af, State: Running, Role: LEADER
I20260808 21:00:22.619521 9166 consensus_queue.cc:237] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } }
I20260808 21:00:22.620466 8482 catalog_manager.cc:5495] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af reported cstate change: term changed from 0 to 1, leader changed from <none> to 316c3a7af34046ad83c9078829ebe2af (127.15.232.131). New cstate: current_term: 1 leader_uuid: "316c3a7af34046ad83c9078829ebe2af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "316c3a7af34046ad83c9078829ebe2af" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 39717 } health_report { overall_health: HEALTHY } } }
I20260808 21:00:22.654513 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.651501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.15.232.131:39717)
0808 21:00:22.651683 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:22.651686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:22.651752 (+ 66us) server_negotiation.cc:407] Connection header received
0808 21:00:22.651852 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.651853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.651879 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.651982 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:00:22.652176 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.652712 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.653597 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.653822 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.653912 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.654132 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:00:22.654246 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:00:22.654354 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.654375 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.654399 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260808 21:00:22.656708 8891 raft_consensus.cc:1261] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [term 1 FOLLOWER]: Refusing update from remote peer 316c3a7af34046ad83c9078829ebe2af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:22.656761 8697 raft_consensus.cc:1261] T 188883482ab147cc83ce3933ba8b5c82 P 10cd42aa62634b96a9ee57abfe0a2d32 [term 1 FOLLOWER]: Refusing update from remote peer 316c3a7af34046ad83c9078829ebe2af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:22.657104 9166 consensus_queue.cc:1035] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [LEADER]: Connected to new peer: Peer: permanent_uuid: "1618cc69088941b1bcf3c47c452e69b9" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:22.657320 9173 consensus_queue.cc:1035] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [LEADER]: Connected to new peer: Peer: permanent_uuid: "10cd42aa62634b96a9ee57abfe0a2d32" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 42931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:22.660581 9178 mvcc.cc:205] Tried to move back new op lower bound from 7316368681598701568 to 7316368681450029056. Current Snapshot: MvccSnapshot[applied={T|T < 7316368681598701568}]
I20260808 21:00:22.661667 9179 mvcc.cc:205] Tried to move back new op lower bound from 7316368681598701568 to 7316368681450029056. Current Snapshot: MvccSnapshot[applied={T|T < 7316368681598701568 or (T in {7316368681598701568})}]
I20260808 21:00:22.663138 8481 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:22.663255 8481 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:22.664161 8481 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5002307d0b5d4495a398063172adaa2d: Sending DeleteTablet for 3 replicas of tablet 188883482ab147cc83ce3933ba8b5c82
I20260808 21:00:22.664556 8866 tablet_service.cc:1514] Processing DeleteTablet for tablet 188883482ab147cc83ce3933ba8b5c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:22 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:57450
I20260808 21:00:22.664551 9044 tablet_service.cc:1514] Processing DeleteTablet for tablet 188883482ab147cc83ce3933ba8b5c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:22 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:42754
I20260808 21:00:22.664870 8672 tablet_service.cc:1514] Processing DeleteTablet for tablet 188883482ab147cc83ce3933ba8b5c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:22 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:41596
I20260808 21:00:22.664968 9195 tablet_replica.cc:330] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: stopping tablet replica
I20260808 21:00:22.665067 9195 raft_consensus.cc:2229] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:22.665179 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 8573
I20260808 21:00:22.675932 9195 raft_consensus.cc:2258] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:22.677186 9195 ts_tablet_manager.cc:1905] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:22.679626 9195 ts_tablet_manager.cc:1918] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:22.679744 9195 log.cc:1196] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/188883482ab147cc83ce3933ba8b5c82
I20260808 21:00:22.680104 9195 ts_tablet_manager.cc:1939] T 188883482ab147cc83ce3933ba8b5c82 P 1618cc69088941b1bcf3c47c452e69b9: Deleting consensus metadata
I20260808 21:00:22.680583 9193 tablet_replica.cc:330] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: stopping tablet replica
I20260808 21:00:22.680640 8461 catalog_manager.cc:4841] TS 1618cc69088941b1bcf3c47c452e69b9 (127.15.232.130:38221): tablet 188883482ab147cc83ce3933ba8b5c82 (table test-table [id=162cff717f17415993a3f0ec1e88907f]) successfully deleted
I20260808 21:00:22.680739 9193 raft_consensus.cc:2229] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 1 LEADER]: Raft consensus shutting down.
I20260808 21:00:22.680855 9193 raft_consensus.cc:2258] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:22.681337 9193 ts_tablet_manager.cc:1905] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:22.683354 9193 ts_tablet_manager.cc:1918] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:22.683475 9193 log.cc:1196] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/188883482ab147cc83ce3933ba8b5c82
I20260808 21:00:22.683881 9193 ts_tablet_manager.cc:1939] T 188883482ab147cc83ce3933ba8b5c82 P 316c3a7af34046ad83c9078829ebe2af: Deleting consensus metadata
I20260808 21:00:22.684405 8463 catalog_manager.cc:4841] TS 316c3a7af34046ad83c9078829ebe2af (127.15.232.131:39717): tablet 188883482ab147cc83ce3933ba8b5c82 (table test-table [id=162cff717f17415993a3f0ec1e88907f]) successfully deleted
W20260808 21:00:22.685761 8461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:42931 (error 108)
I20260808 21:00:22.685982 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 8770
I20260808 21:00:22.704249 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.704097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:42931 (local address 127.0.0.1:41608)
0808 21:00:22.704158 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:22.704195 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:42931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 21:00:22.704411 8461 catalog_manager.cc:4584] TS 10cd42aa62634b96a9ee57abfe0a2d32 (127.15.232.129:42931): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 188883482ab147cc83ce3933ba8b5c82: Network error: Client connection negotiation failed: client connection to 127.15.232.129:42931: connect: Connection refused (error 111)
I20260808 21:00:22.710963 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 8957
I20260808 21:00:22.720939 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 8443
2026-08-08T21:00:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2638 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T21:00:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:22Z Disabled control of system clock
I20260808 21:00:22.757848 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:32891
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:34897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:32891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 21:00:22.849568 9207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:22.849826 9207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:22.849850 9207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:22.851184 9207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:22.851243 9207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:22.851257 9207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:22.851271 9207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:22.851284 9207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:22.853220 9207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:34897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:32891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:32891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:22.853574 9207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:22.854076 9207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:22.860141 9216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:22.860359 9215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:22.860350 9218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:22.860337 9207 server_base.cc:1034] running on GCE node
I20260808 21:00:22.861269 9207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:22.861752 9207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:22.862953 9207 hybrid_clock.cc:648] HybridClock initialized: now 1786222822862888 us; error 80 us; skew 500 ppm
I20260808 21:00:22.868486 9207 webserver.cc:460] Webserver started at http://127.15.232.190:37587/ using document root <none> and password file <none>
I20260808 21:00:22.868947 9207 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:22.869035 9207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:22.869230 9207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:22.870716 9207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "726205dd27124173919e8b6ff9bc2110"
format_stamp: "Formatted at 2026-08-08 21:00:22 on dist-test-slave-c12x"
server_key: "f745a1ddf3b4cbaf39f8233896c7dd57"
server_key_iv: "e59b769c2c7e82bd9943a56680675f89"
server_key_version: "encryptionkey@0"
I20260808 21:00:22.871120 9207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "726205dd27124173919e8b6ff9bc2110"
format_stamp: "Formatted at 2026-08-08 21:00:22 on dist-test-slave-c12x"
server_key: "f745a1ddf3b4cbaf39f8233896c7dd57"
server_key_iv: "e59b769c2c7e82bd9943a56680675f89"
server_key_version: "encryptionkey@0"
I20260808 21:00:22.872793 9207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 21:00:22.873792 9224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:22.874001 9207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:22.874143 9207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "726205dd27124173919e8b6ff9bc2110"
format_stamp: "Formatted at 2026-08-08 21:00:22 on dist-test-slave-c12x"
server_key: "f745a1ddf3b4cbaf39f8233896c7dd57"
server_key_iv: "e59b769c2c7e82bd9943a56680675f89"
server_key_version: "encryptionkey@0"
I20260808 21:00:22.874230 9207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:22.896823 9207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:22.897248 9207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:22.897442 9207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:22.903334 9207 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:32891
I20260808 21:00:22.903359 9306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:32891 every 8 connection(s)
I20260808 21:00:22.903929 9207 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:22.905035 9307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:22.907655 9307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110: Bootstrap starting.
I20260808 21:00:22.908318 9307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:22.908653 9307 log.cc:824] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:22.909523 9307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110: No bootstrap required, opened a new log
I20260808 21:00:22.911237 9307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726205dd27124173919e8b6ff9bc2110" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32891 } }
I20260808 21:00:22.911548 9307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:22.911616 9307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 726205dd27124173919e8b6ff9bc2110, State: Initialized, Role: FOLLOWER
I20260808 21:00:22.911810 9307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726205dd27124173919e8b6ff9bc2110" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32891 } }
I20260808 21:00:22.911913 9307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:22.911939 9307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:22.911989 9307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:22.912607 9307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726205dd27124173919e8b6ff9bc2110" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32891 } }
I20260808 21:00:22.912701 9307 leader_election.cc:302] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 726205dd27124173919e8b6ff9bc2110; no voters:
I20260808 21:00:22.912909 9307 leader_election.cc:288] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:22.912978 9312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:22.913241 9307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:22.913281 9312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [term 1 LEADER]: Becoming Leader. State: Replica: 726205dd27124173919e8b6ff9bc2110, State: Running, Role: LEADER
I20260808 21:00:22.913462 9312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726205dd27124173919e8b6ff9bc2110" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32891 } }
I20260808 21:00:22.913905 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 9207
I20260808 21:00:22.914039 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:22.914054 9315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 726205dd27124173919e8b6ff9bc2110. Latest consensus state: current_term: 1 leader_uuid: "726205dd27124173919e8b6ff9bc2110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726205dd27124173919e8b6ff9bc2110" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32891 } } }
I20260808 21:00:22.914140 9315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:22.914278 16290 external_mini_cluster.cc:1428] Setting key dd6f8bf7d99ee18513d20912bcedf77d
I20260808 21:00:22.914546 9326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:22.914819 9314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "726205dd27124173919e8b6ff9bc2110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726205dd27124173919e8b6ff9bc2110" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32891 } } }
I20260808 21:00:22.915053 9314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:22.915239 9326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:22.917474 9326 catalog_manager.cc:1347] Generated new cluster ID: 904e20dd4168422f80f37be076dd35a2
I20260808 21:00:22.917593 9326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:22.920928 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:22.915068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49064 (local address 127.15.232.190:32891)
0808 21:00:22.915383 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:22.915392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:22.915448 (+ 56us) server_negotiation.cc:407] Connection header received
0808 21:00:22.915590 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:22.915598 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:22.915680 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:22.915842 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:22.916071 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.917406 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.918614 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:22.919191 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:22.919369 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:22.920216 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:22.920220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:22.920221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:22.920411 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:22.920444 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:22.920455 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:22.920578 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:22.920654 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 21:00:22.920741 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":103,"threads_started":1}
I20260808 21:00:22.932562 9326 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:22.933322 9326 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:22.949434 9326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110: Generated new TSK 0
I20260808 21:00:22.949687 9326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:22.953881 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32891
--builtin_ntp_servers=127.15.232.148:34897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 21:00:23.053215 9339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:23.053560 9339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:23.053612 9339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:23.055071 9339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:23.055151 9339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:23.055205 9339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 21:00:23.058599 9339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:34897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:23.059027 9339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:23.059636 9339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:23.067674 9348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:23.067574 9345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:23.067570 9344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:23.067929 9339 server_base.cc:1034] running on GCE node
I20260808 21:00:23.068432 9339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:23.068954 9339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:23.070209 9339 hybrid_clock.cc:648] HybridClock initialized: now 1786222823070191 us; error 52 us; skew 500 ppm
I20260808 21:00:23.071866 9339 webserver.cc:460] Webserver started at http://127.15.232.129:43405/ using document root <none> and password file <none>
I20260808 21:00:23.072165 9339 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:23.072243 9339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:23.072417 9339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:23.073563 9339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "d78d349e2632461a9f0d280ed51e1f5f"
format_stamp: "Formatted at 2026-08-08 21:00:23 on dist-test-slave-c12x"
server_key: "374e18de522879a0c1e5a785dfc25a52"
server_key_iv: "4767d394542c5e3920558717d27bd838"
server_key_version: "encryptionkey@0"
I20260808 21:00:23.073963 9339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "d78d349e2632461a9f0d280ed51e1f5f"
format_stamp: "Formatted at 2026-08-08 21:00:23 on dist-test-slave-c12x"
server_key: "374e18de522879a0c1e5a785dfc25a52"
server_key_iv: "4767d394542c5e3920558717d27bd838"
server_key_version: "encryptionkey@0"
I20260808 21:00:23.075591 9339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 21:00:23.076521 9355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:23.076874 9339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:23.077018 9339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "d78d349e2632461a9f0d280ed51e1f5f"
format_stamp: "Formatted at 2026-08-08 21:00:23 on dist-test-slave-c12x"
server_key: "374e18de522879a0c1e5a785dfc25a52"
server_key_iv: "4767d394542c5e3920558717d27bd838"
server_key_version: "encryptionkey@0"
I20260808 21:00:23.077123 9339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:23.115191 9339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:23.115603 9339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:23.115775 9339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:23.116056 9339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:23.116436 9339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:23.116495 9339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:23.116534 9339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:23.116591 9339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:23.123879 9339 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:35693
I20260808 21:00:23.123963 9511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:35693 every 8 connection(s)
I20260808 21:00:23.124327 9339 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 21:00:23.130295 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.125208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:47437 (local address 127.15.232.190:32891)
0808 21:00:23.125283 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.125286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.125580 (+ 294us) server_negotiation.cc:407] Connection header received
0808 21:00:23.126113 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.126115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.126130 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.126198 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:23.126972 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.127524 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.128601 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.128821 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.128881 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.129932 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:23.129934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:23.129934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:23.129940 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:23.129955 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:23.129959 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:23.130018 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.130190 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.130224 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 21:00:23.130530 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.125200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32891 (local address 127.15.232.129:47437)
0808 21:00:23.125436 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:23.125473 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:23.125988 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:23.126272 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:23.126275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:23.126284 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:23.126855 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.126860 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.127644 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.127645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.128489 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.128492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.128919 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.128919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.129057 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.129754 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 21:00:23.129754 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:23.129787 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 21:00:23.129816 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:23.130088 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:23.130095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:23.130099 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:23.130176 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 21:00:23.130280 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
I20260808 21:00:23.131304 9512 heartbeater.cc:347] Connected to a master server at 127.15.232.190:32891
I20260808 21:00:23.131449 9512 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:23.131742 9512 heartbeater.cc:511] Master 127.15.232.190:32891 requested a full tablet report, sending...
I20260808 21:00:23.131734 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 9339
I20260808 21:00:23.131850 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 21:00:23.132081 16290 external_mini_cluster.cc:1428] Setting key 1d6432f47802538aebcf8daff5e87078
I20260808 21:00:23.132381 9243 ts_manager.cc:194] Registered new tserver with Master: d78d349e2632461a9f0d280ed51e1f5f (127.15.232.129:35693)
I20260808 21:00:23.133294 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32891
--builtin_ntp_servers=127.15.232.148:34897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 21:00:23.133303 9243 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.129:47437
W20260808 21:00:23.235980 9519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:23.236377 9519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:23.236455 9519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:23.239455 9519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:23.239552 9519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:23.239630 9519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 21:00:23.242702 9519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:34897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:23.243031 9519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:23.243376 9519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:23.249533 9526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:23.249608 9527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:23.249608 9529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:23.249958 9519 server_base.cc:1034] running on GCE node
I20260808 21:00:23.250259 9519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:23.250588 9519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:23.251811 9519 hybrid_clock.cc:648] HybridClock initialized: now 1786222823251789 us; error 41 us; skew 500 ppm
I20260808 21:00:23.253693 9519 webserver.cc:460] Webserver started at http://127.15.232.130:45209/ using document root <none> and password file <none>
I20260808 21:00:23.254009 9519 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:23.254094 9519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:23.254259 9519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:23.255501 9519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "678eae5805964b59bc798cf7d2b53810"
format_stamp: "Formatted at 2026-08-08 21:00:23 on dist-test-slave-c12x"
server_key: "44fb3f94055e7ca736abfaef10580d91"
server_key_iv: "55bcf903bc932e07266ec542885a84b8"
server_key_version: "encryptionkey@0"
I20260808 21:00:23.256055 9519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "678eae5805964b59bc798cf7d2b53810"
format_stamp: "Formatted at 2026-08-08 21:00:23 on dist-test-slave-c12x"
server_key: "44fb3f94055e7ca736abfaef10580d91"
server_key_iv: "55bcf903bc932e07266ec542885a84b8"
server_key_version: "encryptionkey@0"
I20260808 21:00:23.257622 9519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:00:23.258572 9539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:23.258796 9519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:23.258906 9519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "678eae5805964b59bc798cf7d2b53810"
format_stamp: "Formatted at 2026-08-08 21:00:23 on dist-test-slave-c12x"
server_key: "44fb3f94055e7ca736abfaef10580d91"
server_key_iv: "55bcf903bc932e07266ec542885a84b8"
server_key_version: "encryptionkey@0"
I20260808 21:00:23.258996 9519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:23.269191 9519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:23.269563 9519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:23.269729 9519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:23.270040 9519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:23.270447 9519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:23.270509 9519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:23.270553 9519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:23.270581 9519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:23.277446 9519 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:43753
I20260808 21:00:23.277489 9684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:43753 every 8 connection(s)
I20260808 21:00:23.277948 9519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 21:00:23.280308 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 9519
I20260808 21:00:23.280455 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 21:00:23.280714 16290 external_mini_cluster.cc:1428] Setting key 6ed115be2f74568d1c81d0c53a7227bb
I20260808 21:00:23.281899 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32891
--builtin_ntp_servers=127.15.232.148:34897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 21:00:23.285264 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.278874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:37769 (local address 127.15.232.190:32891)
0808 21:00:23.278933 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.278935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.279193 (+ 258us) server_negotiation.cc:407] Connection header received
0808 21:00:23.279851 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.279853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.279868 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.279925 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:23.280935 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.281771 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.283341 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.283734 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.283830 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.284943 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:23.284945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:23.284945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:23.284951 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:23.284967 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:23.284972 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:23.285037 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.285162 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.285193 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 21:00:23.285672 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.278830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32891 (local address 127.15.232.130:37769)
0808 21:00:23.279056 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:23.279092 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:23.279724 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:23.280015 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:23.280018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:23.280050 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:23.280762 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.280771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.281890 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.281891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.283229 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.283232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.283892 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.283894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.284092 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.284760 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 21:00:23.284761 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:23.284806 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 21:00:23.284840 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:23.285057 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:23.285064 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:23.285068 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:23.285156 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 21:00:23.285286 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260808 21:00:23.286345 9685 heartbeater.cc:347] Connected to a master server at 127.15.232.190:32891
I20260808 21:00:23.286509 9685 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:23.286736 9685 heartbeater.cc:511] Master 127.15.232.190:32891 requested a full tablet report, sending...
I20260808 21:00:23.287210 9243 ts_manager.cc:194] Registered new tserver with Master: 678eae5805964b59bc798cf7d2b53810 (127.15.232.130:43753)
I20260808 21:00:23.287679 9243 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.130:37769
W20260808 21:00:23.373471 9691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:23.373769 9691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:23.373808 9691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:23.375047 9691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:23.375123 9691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:23.375169 9691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 21:00:23.376722 9691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:34897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:23.377043 9691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:23.377365 9691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:23.382966 9705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:23.382957 9702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:23.382957 9700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:23.383363 9691 server_base.cc:1034] running on GCE node
I20260808 21:00:23.383643 9691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:23.383885 9691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:23.385049 9691 hybrid_clock.cc:648] HybridClock initialized: now 1786222823385027 us; error 46 us; skew 500 ppm
I20260808 21:00:23.386579 9691 webserver.cc:460] Webserver started at http://127.15.232.131:39589/ using document root <none> and password file <none>
I20260808 21:00:23.386857 9691 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:23.386910 9691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:23.387061 9691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:23.388394 9691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6"
format_stamp: "Formatted at 2026-08-08 21:00:23 on dist-test-slave-c12x"
server_key: "dfcda28b105e3c40efefd67e4834432b"
server_key_iv: "054506c8c786921df6c663d73c988d84"
server_key_version: "encryptionkey@0"
I20260808 21:00:23.388839 9691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6"
format_stamp: "Formatted at 2026-08-08 21:00:23 on dist-test-slave-c12x"
server_key: "dfcda28b105e3c40efefd67e4834432b"
server_key_iv: "054506c8c786921df6c663d73c988d84"
server_key_version: "encryptionkey@0"
I20260808 21:00:23.390439 9691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 21:00:23.391330 9712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:23.391577 9691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:23.391705 9691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6"
format_stamp: "Formatted at 2026-08-08 21:00:23 on dist-test-slave-c12x"
server_key: "dfcda28b105e3c40efefd67e4834432b"
server_key_iv: "054506c8c786921df6c663d73c988d84"
server_key_version: "encryptionkey@0"
I20260808 21:00:23.391796 9691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:23.410854 9691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:23.411242 9691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:23.411405 9691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:23.411739 9691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:23.412115 9691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:23.412150 9691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:23.412196 9691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:23.412223 9691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:23.418681 9691 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:43321
I20260808 21:00:23.418731 9864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:43321 every 8 connection(s)
I20260808 21:00:23.419166 9691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 21:00:23.420020 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 9691
I20260808 21:00:23.420176 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 21:00:23.420456 16290 external_mini_cluster.cc:1428] Setting key f5e788a13a74166ac5c5fc54621e6901
I20260808 21:00:23.425813 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.420114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:49119 (local address 127.15.232.190:32891)
0808 21:00:23.420183 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.420186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.420580 (+ 394us) server_negotiation.cc:407] Connection header received
0808 21:00:23.421351 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.421352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.421366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.421420 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:23.422276 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.422864 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.424095 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.424317 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.424433 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.425445 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:23.425447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:23.425447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:23.425454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:23.425470 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:23.425475 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:23.425555 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.425677 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.425716 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 21:00:23.426090 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.420099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32891 (local address 127.15.232.131:49119)
0808 21:00:23.420336 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:23.420393 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:23.421240 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:23.421497 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:23.421502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:23.421513 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:23.422165 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.422171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.422989 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.422990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.423972 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.423975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.424452 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.424453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.424611 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.425269 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 21:00:23.425270 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:23.425304 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 21:00:23.425332 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:23.425572 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:23.425578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:23.425581 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 21:00:23.425679 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 21:00:23.425803 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":91,"threads_started":1}
I20260808 21:00:23.426611 9865 heartbeater.cc:347] Connected to a master server at 127.15.232.190:32891
I20260808 21:00:23.426709 9865 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:23.426954 9865 heartbeater.cc:511] Master 127.15.232.190:32891 requested a full tablet report, sending...
I20260808 21:00:23.427376 9243 ts_manager.cc:194] Registered new tserver with Master: 7ba9c4d6bec04aef9ecbf8914dc2a7d6 (127.15.232.131:43321)
I20260808 21:00:23.427802 9243 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.131:49119
I20260808 21:00:23.432546 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 21:00:23.437345 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.434082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.15.232.190:32891)
0808 21:00:23.434158 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.434161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.434307 (+ 146us) server_negotiation.cc:407] Connection header received
0808 21:00:23.434340 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.434344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.434359 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.434425 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:23.434612 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.435165 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.436107 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.436297 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.436363 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.437109 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:23.437111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:23.437111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:23.437115 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:23.437130 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:23.437135 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:23.437194 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.437249 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.437278 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 21:00:23.439502 9243 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 21:00:23.441147 9243 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 21:00:23.448797 9876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.443934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35693 (local address 127.0.0.1:53184)
0808 21:00:23.444101 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:23.444142 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:23.444339 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:23.444973 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:23.444976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:23.444978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:23.445073 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.445076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.446174 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.446175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.447420 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.447423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.448443 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.448445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.448545 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.448561 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 21:00:23.448624 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 21:00:23.448701 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":115,"threads_started":1}
I20260808 21:00:23.448809 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.444045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53184 (local address 127.15.232.129:35693)
0808 21:00:23.444276 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.444285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.444329 (+ 44us) server_negotiation.cc:407] Connection header received
0808 21:00:23.444445 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.444448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.444465 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.444877 (+ 412us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:23.445175 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.446091 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.447547 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.448374 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.448448 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.448615 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.448659 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.448704 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":131,"threads_started":1}
I20260808 21:00:23.448812 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.444472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48416 (local address 127.15.232.130:43753)
0808 21:00:23.444878 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.444883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.444893 (+ 10us) server_negotiation.cc:407] Connection header received
0808 21:00:23.444933 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.444935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.444952 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.445050 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:23.445225 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.446492 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.447766 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.448286 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.448387 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.448485 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.448682 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.448722 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":96,"threads_started":1}
I20260808 21:00:23.449425 9881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.444506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:43321 (local address 127.0.0.1:34036)
0808 21:00:23.445153 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:23.445165 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:23.445229 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:23.445642 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:23.445643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:23.445645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:23.445700 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.445702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.447081 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.447082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.448243 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.448247 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.449185 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.449186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.449246 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.449250 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:23.449307 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 21:00:23.449352 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":55,"threads_started":1}
I20260808 21:00:23.449628 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.444655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.15.232.131:43321)
0808 21:00:23.444989 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.444993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.445422 (+ 429us) server_negotiation.cc:407] Connection header received
0808 21:00:23.445513 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.445516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.445533 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.445625 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:23.445783 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.446983 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.448350 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.449065 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.449182 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.449267 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.449322 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.449358 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":105,"threads_started":1}
I20260808 21:00:23.449659 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.444339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:43753 (local address 127.0.0.1:48416)
0808 21:00:23.444656 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:23.444667 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:23.444829 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:23.445074 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:23.445075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:23.445076 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:23.445130 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.445132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.446647 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.446648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.447673 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.447675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.448523 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.448524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.448584 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.448588 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:23.448640 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 21:00:23.449415 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":715,"thread_start_us":59,"threads_started":1}
I20260808 21:00:23.450237 9776 tablet_service.cc:1467] Processing CreateTablet for tablet 08f3c7fea1bd48cbad845fc9f7311e21 (DEFAULT_TABLE table=test-table [id=aae9d73924cd408eb603a999624b25b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:23.450188 9422 tablet_service.cc:1467] Processing CreateTablet for tablet 08f3c7fea1bd48cbad845fc9f7311e21 (DEFAULT_TABLE table=test-table [id=aae9d73924cd408eb603a999624b25b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:23.450623 9776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f3c7fea1bd48cbad845fc9f7311e21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:23.450636 9422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f3c7fea1bd48cbad845fc9f7311e21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:23.450548 9599 tablet_service.cc:1467] Processing CreateTablet for tablet 08f3c7fea1bd48cbad845fc9f7311e21 (DEFAULT_TABLE table=test-table [id=aae9d73924cd408eb603a999624b25b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:23.451014 9599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f3c7fea1bd48cbad845fc9f7311e21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:23.453464 9885 tablet_bootstrap.cc:492] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: Bootstrap starting.
I20260808 21:00:23.454209 9885 tablet_bootstrap.cc:654] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:23.454563 9885 log.cc:824] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:23.454720 9888 tablet_bootstrap.cc:492] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f: Bootstrap starting.
I20260808 21:00:23.455531 9888 tablet_bootstrap.cc:654] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:23.455598 9885 tablet_bootstrap.cc:492] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: No bootstrap required, opened a new log
I20260808 21:00:23.455623 9887 tablet_bootstrap.cc:492] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: Bootstrap starting.
I20260808 21:00:23.455681 9885 ts_tablet_manager.cc:1397] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 21:00:23.455869 9888 log.cc:824] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:23.456284 9887 tablet_bootstrap.cc:654] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:23.456624 9887 log.cc:824] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:23.456909 9888 tablet_bootstrap.cc:492] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f: No bootstrap required, opened a new log
I20260808 21:00:23.456995 9888 ts_tablet_manager.cc:1397] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 21:00:23.457511 9887 tablet_bootstrap.cc:492] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: No bootstrap required, opened a new log
I20260808 21:00:23.457597 9887 ts_tablet_manager.cc:1397] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 21:00:23.458702 9885 raft_consensus.cc:348] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } }
I20260808 21:00:23.458930 9885 raft_consensus.cc:374] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:23.458992 9885 raft_consensus.cc:729] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba9c4d6bec04aef9ecbf8914dc2a7d6, State: Initialized, Role: FOLLOWER
I20260808 21:00:23.459139 9885 consensus_queue.cc:260] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } }
I20260808 21:00:23.459668 9885 ts_tablet_manager.cc:1428] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260808 21:00:23.459699 9865 heartbeater.cc:503] Master 127.15.232.190:32891 was elected leader, sending a full tablet report...
I20260808 21:00:23.460081 9888 raft_consensus.cc:348] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } }
I20260808 21:00:23.460254 9888 raft_consensus.cc:374] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:23.460291 9888 raft_consensus.cc:729] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78d349e2632461a9f0d280ed51e1f5f, State: Initialized, Role: FOLLOWER
I20260808 21:00:23.460209 9887 raft_consensus.cc:348] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } }
I20260808 21:00:23.460347 9887 raft_consensus.cc:374] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:23.460422 9887 raft_consensus.cc:729] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 678eae5805964b59bc798cf7d2b53810, State: Initialized, Role: FOLLOWER
I20260808 21:00:23.460450 9888 consensus_queue.cc:260] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } }
I20260808 21:00:23.460530 9887 consensus_queue.cc:260] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } }
I20260808 21:00:23.460772 9887 ts_tablet_manager.cc:1428] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 21:00:23.460865 9888 ts_tablet_manager.cc:1428] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260808 21:00:23.460855 9685 heartbeater.cc:503] Master 127.15.232.190:32891 was elected leader, sending a full tablet report...
I20260808 21:00:23.461009 9512 heartbeater.cc:503] Master 127.15.232.190:32891 was elected leader, sending a full tablet report...
W20260808 21:00:23.528677 9687 tablet.cc:2369] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 21:00:23.625149 9516 tablet.cc:2369] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 21:00:23.644086 9894 raft_consensus.cc:482] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:23.644304 9894 raft_consensus.cc:504] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } }
I20260808 21:00:23.644963 9894 leader_election.cc:288] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 678eae5805964b59bc798cf7d2b53810 (127.15.232.130:43753), 7ba9c4d6bec04aef9ecbf8914dc2a7d6 (127.15.232.131:43321)
I20260808 21:00:23.648605 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.645292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:46037 (local address 127.15.232.130:43753)
0808 21:00:23.645389 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.645391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.645451 (+ 60us) server_negotiation.cc:407] Connection header received
0808 21:00:23.645597 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.645599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.645615 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.645698 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:23.645948 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.646487 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.647722 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.648153 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.648256 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.648265 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.648455 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.648501 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 21:00:23.648622 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.645610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:56211 (local address 127.15.232.131:43321)
0808 21:00:23.645676 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.645678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.645708 (+ 30us) server_negotiation.cc:407] Connection header received
0808 21:00:23.645792 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.645793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.645802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.645887 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:23.646075 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.646546 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.647800 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.648206 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.648301 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.648310 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.648497 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.648541 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 21:00:23.648648 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.645510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:43321 (local address 127.15.232.129:56211)
0808 21:00:23.645642 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:23.645653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:23.645721 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:23.645893 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:23.645894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:23.645896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:23.645986 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.645989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.646644 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.646645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.647683 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.647686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.648315 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.648316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.648404 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.648408 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:23.648485 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 21:00:23.648528 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":20,"thread_start_us":67,"threads_started":1}
I20260808 21:00:23.648679 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.645216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:43753 (local address 127.15.232.129:46037)
0808 21:00:23.645365 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:23.645378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:23.645458 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:23.645767 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:23.645768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:23.645770 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:23.645853 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.645855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.646608 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.646610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.647594 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:23.647598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.648256 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.648257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:23.648340 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.648345 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 21:00:23.648445 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 21:00:23.648497 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260808 21:00:23.649111 9627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f3c7fea1bd48cbad845fc9f7311e21" candidate_uuid: "d78d349e2632461a9f0d280ed51e1f5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "678eae5805964b59bc798cf7d2b53810" is_pre_election: true
I20260808 21:00:23.649160 9807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f3c7fea1bd48cbad845fc9f7311e21" candidate_uuid: "d78d349e2632461a9f0d280ed51e1f5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" is_pre_election: true
I20260808 21:00:23.649330 9807 raft_consensus.cc:2454] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d78d349e2632461a9f0d280ed51e1f5f in term 0.
I20260808 21:00:23.649331 9627 raft_consensus.cc:2454] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d78d349e2632461a9f0d280ed51e1f5f in term 0.
I20260808 21:00:23.649641 9356 leader_election.cc:302] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba9c4d6bec04aef9ecbf8914dc2a7d6, d78d349e2632461a9f0d280ed51e1f5f; no voters:
I20260808 21:00:23.649801 9894 raft_consensus.cc:2781] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 21:00:23.649869 9894 raft_consensus.cc:482] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 21:00:23.649899 9894 raft_consensus.cc:3037] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:23.650784 9894 raft_consensus.cc:504] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } }
I20260808 21:00:23.650966 9894 leader_election.cc:288] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [CANDIDATE]: Term 1 election: Requested vote from peers 678eae5805964b59bc798cf7d2b53810 (127.15.232.130:43753), 7ba9c4d6bec04aef9ecbf8914dc2a7d6 (127.15.232.131:43321)
I20260808 21:00:23.651194 9807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f3c7fea1bd48cbad845fc9f7311e21" candidate_uuid: "d78d349e2632461a9f0d280ed51e1f5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6"
I20260808 21:00:23.651237 9627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f3c7fea1bd48cbad845fc9f7311e21" candidate_uuid: "d78d349e2632461a9f0d280ed51e1f5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "678eae5805964b59bc798cf7d2b53810"
I20260808 21:00:23.651320 9627 raft_consensus.cc:3037] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:23.651309 9807 raft_consensus.cc:3037] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:23.652186 9627 raft_consensus.cc:2454] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d78d349e2632461a9f0d280ed51e1f5f in term 1.
I20260808 21:00:23.652191 9807 raft_consensus.cc:2454] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d78d349e2632461a9f0d280ed51e1f5f in term 1.
I20260808 21:00:23.652387 9356 leader_election.cc:302] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba9c4d6bec04aef9ecbf8914dc2a7d6, d78d349e2632461a9f0d280ed51e1f5f; no voters:
I20260808 21:00:23.652514 9894 raft_consensus.cc:2781] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:23.652694 9894 raft_consensus.cc:686] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 1 LEADER]: Becoming Leader. State: Replica: d78d349e2632461a9f0d280ed51e1f5f, State: Running, Role: LEADER
I20260808 21:00:23.652827 9894 consensus_queue.cc:237] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } }
I20260808 21:00:23.653889 9246 catalog_manager.cc:5495] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f reported cstate change: term changed from 0 to 1, leader changed from <none> to d78d349e2632461a9f0d280ed51e1f5f (127.15.232.129). New cstate: current_term: 1 leader_uuid: "d78d349e2632461a9f0d280ed51e1f5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d78d349e2632461a9f0d280ed51e1f5f" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 35693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 } health_report { overall_health: UNKNOWN } } }
W20260808 21:00:23.669814 9867 tablet.cc:2369] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 21:00:23.681269 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.678496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53194 (local address 127.15.232.129:35693)
0808 21:00:23.678565 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:23.678567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:23.678576 (+ 9us) server_negotiation.cc:407] Connection header received
0808 21:00:23.678632 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:23.678633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:23.678654 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:23.678717 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:00:23.678956 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.679482 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.680339 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:23.680586 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:23.680671 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:23.680856 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:00:23.680985 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:00:23.681085 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:23.681142 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 21:00:23.681166 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 21:00:23.683311 9807 raft_consensus.cc:1261] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [term 1 FOLLOWER]: Refusing update from remote peer d78d349e2632461a9f0d280ed51e1f5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:23.683331 9627 raft_consensus.cc:1261] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [term 1 FOLLOWER]: Refusing update from remote peer d78d349e2632461a9f0d280ed51e1f5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:23.683825 9894 consensus_queue.cc:1035] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "678eae5805964b59bc798cf7d2b53810" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 43753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:23.684224 9898 consensus_queue.cc:1035] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba9c4d6bec04aef9ecbf8914dc2a7d6" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:23.689826 9246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:23.689975 9246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:23.690886 9246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 726205dd27124173919e8b6ff9bc2110: Sending DeleteTablet for 3 replicas of tablet 08f3c7fea1bd48cbad845fc9f7311e21
I20260808 21:00:23.691236 9422 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f3c7fea1bd48cbad845fc9f7311e21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:23 UTC) from {username='slave'} at 127.0.0.1:53184
I20260808 21:00:23.691361 9776 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f3c7fea1bd48cbad845fc9f7311e21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:23 UTC) from {username='slave'} at 127.0.0.1:34036
I20260808 21:00:23.691361 9599 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f3c7fea1bd48cbad845fc9f7311e21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:23 UTC) from {username='slave'} at 127.0.0.1:48416
I20260808 21:00:23.691643 9914 tablet_replica.cc:330] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: stopping tablet replica
I20260808 21:00:23.691740 9913 tablet_replica.cc:330] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f: stopping tablet replica
I20260808 21:00:23.691771 9914 raft_consensus.cc:2229] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:23.691823 9913 raft_consensus.cc:2229] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 1 LEADER]: Raft consensus shutting down.
I20260808 21:00:23.691900 9914 raft_consensus.cc:2258] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:23.691924 9913 raft_consensus.cc:2258] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:23.692185 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 9339
I20260808 21:00:23.692399 9914 ts_tablet_manager.cc:1905] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:23.692435 9913 ts_tablet_manager.cc:1905] T 08f3c7fea1bd48cbad845fc9f7311e21 P d78d349e2632461a9f0d280ed51e1f5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:23.693850 9915 tablet_replica.cc:330] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: stopping tablet replica
I20260808 21:00:23.694034 9914 ts_tablet_manager.cc:1918] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:23.694032 9915 raft_consensus.cc:2229] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:23.694104 9914 log.cc:1196] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/08f3c7fea1bd48cbad845fc9f7311e21
I20260808 21:00:23.694170 9915 raft_consensus.cc:2258] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:23.694415 9914 ts_tablet_manager.cc:1939] T 08f3c7fea1bd48cbad845fc9f7311e21 P 678eae5805964b59bc798cf7d2b53810: Deleting consensus metadata
I20260808 21:00:23.694846 9226 catalog_manager.cc:4841] TS 678eae5805964b59bc798cf7d2b53810 (127.15.232.130:43753): tablet 08f3c7fea1bd48cbad845fc9f7311e21 (table test-table [id=aae9d73924cd408eb603a999624b25b2]) successfully deleted
I20260808 21:00:23.695542 9915 ts_tablet_manager.cc:1905] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:23.696897 9915 ts_tablet_manager.cc:1918] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:23.696951 9915 log.cc:1196] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/08f3c7fea1bd48cbad845fc9f7311e21
I20260808 21:00:23.697142 9915 ts_tablet_manager.cc:1939] T 08f3c7fea1bd48cbad845fc9f7311e21 P 7ba9c4d6bec04aef9ecbf8914dc2a7d6: Deleting consensus metadata
I20260808 21:00:23.697827 9226 catalog_manager.cc:4841] TS 7ba9c4d6bec04aef9ecbf8914dc2a7d6 (127.15.232.131:43321): tablet 08f3c7fea1bd48cbad845fc9f7311e21 (table test-table [id=aae9d73924cd408eb603a999624b25b2]) successfully deleted
W20260808 21:00:23.704996 9226 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:35693 (error 108)
I20260808 21:00:23.705672 9881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:23.705496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:35693 (local address 127.0.0.1:53200)
0808 21:00:23.705578 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:23.705629 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:35693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 21:00:23.705734 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 9519
W20260808 21:00:23.705832 9226 catalog_manager.cc:4584] TS d78d349e2632461a9f0d280ed51e1f5f (127.15.232.129:35693): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 08f3c7fea1bd48cbad845fc9f7311e21: Network error: Client connection negotiation failed: client connection to 127.15.232.129:35693: connect: Connection refused (error 111)
I20260808 21:00:23.723377 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 9691
I20260808 21:00:23.735586 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 9207
2026-08-08T21:00:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1015 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:23 dist-test-slave-c12x krb5kdc[9922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:23 dist-test-slave-c12x krb5kdc[9922](info): set up 2 sockets
Aug 08 21:00:23 dist-test-slave-c12x krb5kdc[9922](info): commencing operation
krb5kdc: starting...
W20260808 21:00:25.016695 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:25 dist-test-slave-c12x krb5kdc[9922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:00:25.159792 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:32951
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:32951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:25.251545 9938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:25.251850 9938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:25.251896 9938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:25.253165 9938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:25.253255 9938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:25.253281 9938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:25.253311 9938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:25.253340 9938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:25.255226 9938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:32951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:32951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:25.255584 9938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:25.255852 9938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:25.261348 9943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:25.261369 9950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:25.261425 9947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:25.261931 9938 server_base.cc:1034] running on GCE node
I20260808 21:00:25.262213 9938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:25.262535 9938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:25.263780 9938 hybrid_clock.cc:648] HybridClock initialized: now 1786222825263767 us; error 33 us; skew 500 ppm
Aug 08 21:00:25 dist-test-slave-c12x krb5kdc[9922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222825, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:25.266772 9938 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:25.267355 9938 webserver.cc:460] Webserver started at http://127.15.232.190:33393/ using document root <none> and password file <none>
I20260808 21:00:25.267642 9938 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:25.267724 9938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:25.267915 9938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:25.269178 9938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "d66db36ece3c49db9fee228310ae6142"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "5c338014f4b4c0109713cc75729925cd"
server_key_iv: "bee61e034ba61d3d34c58437f9bf2ed3"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.269603 9938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "d66db36ece3c49db9fee228310ae6142"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "5c338014f4b4c0109713cc75729925cd"
server_key_iv: "bee61e034ba61d3d34c58437f9bf2ed3"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.271185 9938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 21:00:25.272140 9960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.272362 9938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:25.272482 9938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "d66db36ece3c49db9fee228310ae6142"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "5c338014f4b4c0109713cc75729925cd"
server_key_iv: "bee61e034ba61d3d34c58437f9bf2ed3"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.272570 9938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:25.289762 9938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:25.292425 9938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:25.292644 9938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:25.297820 9938 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:32951
I20260808 21:00:25.297922 10038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:32951 every 8 connection(s)
I20260808 21:00:25.298345 9938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:25.299423 10039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:25.302033 10039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142: Bootstrap starting.
I20260808 21:00:25.302716 10039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:25.303089 10039 log.cc:824] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:25.304075 10039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142: No bootstrap required, opened a new log
I20260808 21:00:25.306013 10039 raft_consensus.cc:348] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d66db36ece3c49db9fee228310ae6142" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32951 } }
I20260808 21:00:25.306181 10039 raft_consensus.cc:374] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:25.306221 10039 raft_consensus.cc:729] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d66db36ece3c49db9fee228310ae6142, State: Initialized, Role: FOLLOWER
I20260808 21:00:25.306416 10039 consensus_queue.cc:260] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d66db36ece3c49db9fee228310ae6142" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32951 } }
I20260808 21:00:25.306499 10039 raft_consensus.cc:388] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:25.306545 10039 raft_consensus.cc:482] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:25.306600 10039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:25.306921 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 9938
I20260808 21:00:25.307067 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:25.307298 16290 external_mini_cluster.cc:1428] Setting key 7619aa3ede9eea3abd39e65f58b30fe7
I20260808 21:00:25.307463 10039 raft_consensus.cc:504] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d66db36ece3c49db9fee228310ae6142" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32951 } }
I20260808 21:00:25.307616 10039 leader_election.cc:302] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d66db36ece3c49db9fee228310ae6142; no voters:
I20260808 21:00:25.307858 10039 leader_election.cc:288] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:25.308084 10044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:25.308192 10039 sys_catalog.cc:564] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:25.308261 10044 raft_consensus.cc:686] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [term 1 LEADER]: Becoming Leader. State: Replica: d66db36ece3c49db9fee228310ae6142, State: Running, Role: LEADER
I20260808 21:00:25.308367 10044 consensus_queue.cc:237] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d66db36ece3c49db9fee228310ae6142" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32951 } }
I20260808 21:00:25.309202 10046 sys_catalog.cc:455] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d66db36ece3c49db9fee228310ae6142. Latest consensus state: current_term: 1 leader_uuid: "d66db36ece3c49db9fee228310ae6142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d66db36ece3c49db9fee228310ae6142" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32951 } } }
I20260808 21:00:25.309409 10042 sys_catalog.cc:455] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d66db36ece3c49db9fee228310ae6142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d66db36ece3c49db9fee228310ae6142" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 32951 } } }
I20260808 21:00:25.309518 10046 sys_catalog.cc:458] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:25.309634 10042 sys_catalog.cc:458] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:25.309963 10058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:25.310602 10058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:25.312433 10058 catalog_manager.cc:1347] Generated new cluster ID: f5e51d84600f4ea8b18083e7f04bbad0
I20260808 21:00:25.312495 10058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 21:00:25 dist-test-slave-c12x krb5kdc[9922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:25.319871 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.307946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.15.232.190:32951)
0808 21:00:25.309025 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.309035 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.309070 (+ 35us) server_negotiation.cc:407] Connection header received
0808 21:00:25.309213 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.309221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.309282 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.310526 (+ 1244us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:25.310608 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.311799 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.312864 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.313309 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.313432 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.315890 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:25.315894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:25.315895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:25.315907 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:25.318153 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.318552 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.318556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.318557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.318682 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.318903 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.318904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.318905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.319173 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:25.319355 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.319500 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.319657 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":983,"thread_start_us":98,"threads_started":1}
I20260808 21:00:25.321295 10058 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:25.321817 10058 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:25.324867 10058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142: Generated new TSK 0
I20260808 21:00:25.325127 10058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 21:00:25.362886 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32951
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:25.468452 10069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:25.468735 10069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:25.468780 10069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:25.470072 10069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:25.470147 10069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:25.470198 10069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 21:00:25.471921 10069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:25.472210 10069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:25.472467 10069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:25.478129 10078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:25.478133 10080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:25.478127 10077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:25.478437 10069 server_base.cc:1034] running on GCE node
I20260808 21:00:25.478683 10069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:25.478971 10069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:25.480180 10069 hybrid_clock.cc:648] HybridClock initialized: now 1786222825480167 us; error 36 us; skew 500 ppm
Aug 08 21:00:25 dist-test-slave-c12x krb5kdc[9922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222825, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:25.483214 10069 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 21:00:25.483786 10069 webserver.cc:460] Webserver started at http://127.15.232.129:43025/ using document root <none> and password file <none>
I20260808 21:00:25.484031 10069 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:25.484124 10069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:25.484287 10069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:25.485412 10069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "585059f6d61f4a8eb987b35871326326"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "3a1ecb95786654ad4cdff4ad642005d9"
server_key_iv: "41050fec5b6b0a415d5c7ffe2f2d1b09"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.485816 10069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "585059f6d61f4a8eb987b35871326326"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "3a1ecb95786654ad4cdff4ad642005d9"
server_key_iv: "41050fec5b6b0a415d5c7ffe2f2d1b09"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.487646 10069 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 21:00:25.488533 10090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.488811 10069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:25.488937 10069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "585059f6d61f4a8eb987b35871326326"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "3a1ecb95786654ad4cdff4ad642005d9"
server_key_iv: "41050fec5b6b0a415d5c7ffe2f2d1b09"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.489037 10069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:25.506402 10069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:25.509204 10069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:25.509411 10069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:25.509825 10069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:25.510257 10069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:25.510329 10069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.510376 10069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:25.510408 10069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.518828 10069 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:34493
I20260808 21:00:25.518887 10253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:34493 every 8 connection(s)
I20260808 21:00:25.519243 10069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 21:00:25.520591 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 10069
I20260808 21:00:25.520748 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 21:00:25.520979 16290 external_mini_cluster.cc:1428] Setting key 1034e1bf524c7e8766f5de874e0a2ff3
Aug 08 21:00:25 dist-test-slave-c12x krb5kdc[9922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222825, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
I20260808 21:00:25.532727 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.520166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:48489 (local address 127.15.232.190:32951)
0808 21:00:25.520261 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.520264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.520652 (+ 388us) server_negotiation.cc:407] Connection header received
0808 21:00:25.521397 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.521398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.521436 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.521509 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:25.522798 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.523676 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.525397 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.525748 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.525839 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.528676 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:25.528678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:25.528678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:25.528689 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:25.530807 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.531449 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.531451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.531451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.531525 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.531853 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.531854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.531854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.532040 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:25.532136 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.532480 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.532586 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 21:00:25.532871 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.520065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32951 (local address 127.15.232.129:48489)
0808 21:00:25.520323 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:25.520378 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:25.521268 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:25.521586 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:25.521590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:25.522457 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:25.522669 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.522674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.523804 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.523805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.525253 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.525257 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.525861 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.525862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.525986 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.526762 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:25.526772 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:25.528549 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:25.530967 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:25.530978 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:25.530980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:25.531316 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:25.531620 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:25.531622 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:25.531622 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:25.531759 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:25.532134 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:25.532141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:25.532295 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 21:00:25.532434 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 21:00:25.532593 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":108,"threads_started":1}
I20260808 21:00:25.533756 10255 heartbeater.cc:347] Connected to a master server at 127.15.232.190:32951
I20260808 21:00:25.533962 10255 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:25.534226 10255 heartbeater.cc:511] Master 127.15.232.190:32951 requested a full tablet report, sending...
I20260808 21:00:25.534883 9984 ts_manager.cc:194] Registered new tserver with Master: 585059f6d61f4a8eb987b35871326326 (127.15.232.129:34493)
I20260808 21:00:25.535661 9984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:48489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 21:00:25.557216 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32951
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:25.651265 10263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:25.651533 10263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:25.651562 10263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:25.652767 10263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:25.652817 10263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:25.652846 10263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 21:00:25.654516 10263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:25.654733 10263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:25.654937 10263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:25.660360 10270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:25.660374 10275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:25.660328 10269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:25.660714 10263 server_base.cc:1034] running on GCE node
I20260808 21:00:25.660957 10263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:25.661199 10263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:25.662380 10263 hybrid_clock.cc:648] HybridClock initialized: now 1786222825662364 us; error 36 us; skew 500 ppm
Aug 08 21:00:25 dist-test-slave-c12x krb5kdc[9922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222825, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:25.665666 10263 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 21:00:25.666296 10263 webserver.cc:460] Webserver started at http://127.15.232.130:42873/ using document root <none> and password file <none>
I20260808 21:00:25.666572 10263 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:25.666653 10263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:25.666826 10263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:25.668179 10263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "0462e398d32cfd2a2d6c0e5b3e9dbf6e"
server_key_iv: "2498f1a44a1283da5a789c8cbb431a54"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.668618 10263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "0462e398d32cfd2a2d6c0e5b3e9dbf6e"
server_key_iv: "2498f1a44a1283da5a789c8cbb431a54"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.670282 10263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:00:25.671340 10284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.671591 10263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:25.671728 10263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "0462e398d32cfd2a2d6c0e5b3e9dbf6e"
server_key_iv: "2498f1a44a1283da5a789c8cbb431a54"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.671823 10263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:25.695510 10263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:25.698026 10263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:25.698236 10263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:25.698518 10263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:25.699061 10263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:25.699153 10263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.699206 10263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:25.699239 10263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.706512 10263 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:38707
I20260808 21:00:25.706557 10442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:38707 every 8 connection(s)
I20260808 21:00:25.706952 10263 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
Aug 08 21:00:25 dist-test-slave-c12x krb5kdc[9922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222825, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:25.714938 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 10263
I20260808 21:00:25.715097 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 21:00:25.715344 16290 external_mini_cluster.cc:1428] Setting key 2e48c9b2f906d7000746247114b79544
I20260808 21:00:25.719118 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.708114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:60639 (local address 127.15.232.190:32951)
0808 21:00:25.708192 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.708194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.708460 (+ 266us) server_negotiation.cc:407] Connection header received
0808 21:00:25.709043 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.709044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.709056 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.709149 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:25.710352 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.710908 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.712050 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.712269 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.712339 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.714886 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:25.714889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:25.714889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:25.714901 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:25.717157 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.717678 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.717679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.717680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.717755 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.718041 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.718042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.718043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.718211 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:25.718290 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.718943 (+ 653us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.719042 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 21:00:25.719517 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.708078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32951 (local address 127.15.232.130:60639)
0808 21:00:25.708275 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:25.708319 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:25.708919 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:25.709255 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:25.709259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:25.710015 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:25.710225 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.710229 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.711037 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.711038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.711949 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.711952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.712378 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.712380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.712548 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.713156 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:25.713165 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:25.714765 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:25.717273 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:25.717283 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:25.717286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:25.717591 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:25.717834 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:25.717834 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:25.717835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:25.717964 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:25.718332 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:25.718343 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:25.718585 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 21:00:25.718890 (+ 305us) client_negotiation.cc:240] Negotiation successful
0808 21:00:25.719143 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":120,"threads_started":1}
I20260808 21:00:25.720235 10443 heartbeater.cc:347] Connected to a master server at 127.15.232.190:32951
I20260808 21:00:25.720360 10443 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:25.720605 10443 heartbeater.cc:511] Master 127.15.232.190:32951 requested a full tablet report, sending...
I20260808 21:00:25.721099 9984 ts_manager.cc:194] Registered new tserver with Master: 4d2d93029ab6469a9ca4c6692a6aa3ad (127.15.232.130:38707)
I20260808 21:00:25.721513 9984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:60639
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 21:00:25.750366 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32951
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:25.843341 10451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:25.843757 10451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:25.843802 10451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:25.845064 10451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:25.845115 10451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:25.845144 10451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 21:00:25.846737 10451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:32951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:25.847007 10451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:25.847282 10451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:25.852802 10459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:25.852789 10458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:25.852794 10461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:25.853165 10451 server_base.cc:1034] running on GCE node
I20260808 21:00:25.853438 10451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:25.853978 10451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:25.855208 10451 hybrid_clock.cc:648] HybridClock initialized: now 1786222825855193 us; error 68 us; skew 500 ppm
Aug 08 21:00:25 dist-test-slave-c12x krb5kdc[9922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222825, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:25.858318 10451 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 21:00:25.858858 10451 webserver.cc:460] Webserver started at http://127.15.232.131:41747/ using document root <none> and password file <none>
I20260808 21:00:25.859098 10451 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:25.859146 10451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:25.859313 10451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:25.860673 10451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "07cb4d5fa55849c4901f86e4130b081e"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "0ce16267cf01aeefc533f6a47d7f0cd9"
server_key_iv: "36ddf3ad40169ac398350067786b8816"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.861320 10451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "07cb4d5fa55849c4901f86e4130b081e"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "0ce16267cf01aeefc533f6a47d7f0cd9"
server_key_iv: "36ddf3ad40169ac398350067786b8816"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.862874 10451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 21:00:25.863996 10469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.864324 10451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 21:00:25.864463 10451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "07cb4d5fa55849c4901f86e4130b081e"
format_stamp: "Formatted at 2026-08-08 21:00:25 on dist-test-slave-c12x"
server_key: "0ce16267cf01aeefc533f6a47d7f0cd9"
server_key_iv: "36ddf3ad40169ac398350067786b8816"
server_key_version: "encryptionkey@0"
I20260808 21:00:25.864594 10451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:25.877347 10451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:25.880877 10451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:25.881084 10451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:25.881409 10451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:25.881856 10451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:25.881933 10451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.881974 10451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:25.882002 10451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:25.889534 10451 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:38589
I20260808 21:00:25.889590 10632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:38589 every 8 connection(s)
I20260808 21:00:25.890029 10451 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 21:00:25.897506 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 10451
Aug 08 21:00:25 dist-test-slave-c12x krb5kdc[9922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222825, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:25.897687 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 21:00:25.897902 16290 external_mini_cluster.cc:1428] Setting key 26cb484de52b84c5ef19dc8e575526f3
I20260808 21:00:25.902087 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.891031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:47277 (local address 127.15.232.190:32951)
0808 21:00:25.891092 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.891095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.891236 (+ 141us) server_negotiation.cc:407] Connection header received
0808 21:00:25.891966 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.891967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.891980 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.892052 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:25.893196 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.893776 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.895300 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.895670 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.895755 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.898261 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:25.898262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:25.898262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:25.898269 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:25.900193 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.900789 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.900791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.900792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.900925 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.901296 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.901298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.901298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.901464 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:25.901591 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.901919 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.902005 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 21:00:25.902261 10636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.890918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:32951 (local address 127.15.232.131:47277)
0808 21:00:25.891095 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:25.891139 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:25.891783 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:25.892137 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:25.892141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:25.892766 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:25.893048 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.893053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.893891 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.893892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.895169 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.895173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.895787 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.895789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.895904 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.896466 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:25.896475 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:25.898088 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:25.900287 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:25.900295 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:25.900297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:25.900602 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:25.901043 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:25.901044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:25.901044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:25.901178 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:25.901595 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:25.901602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:25.901745 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 21:00:25.901852 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 21:00:25.902015 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260808 21:00:25.902916 10633 heartbeater.cc:347] Connected to a master server at 127.15.232.190:32951
I20260808 21:00:25.903029 10633 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:25.903208 10633 heartbeater.cc:511] Master 127.15.232.190:32951 requested a full tablet report, sending...
I20260808 21:00:25.903666 9984 ts_manager.cc:194] Registered new tserver with Master: 07cb4d5fa55849c4901f86e4130b081e (127.15.232.131:38589)
I20260808 21:00:25.904036 9984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:47277
I20260808 21:00:25.910387 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 21:00:25.919155 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.911945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34894 (local address 127.15.232.190:32951)
0808 21:00:25.912070 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.912073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.912158 (+ 85us) server_negotiation.cc:407] Connection header received
0808 21:00:25.912259 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.912260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.912270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.912329 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:25.912971 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.913481 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.914214 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.914414 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.914492 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.916101 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:25.916102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:25.916103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:25.916111 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:25.917630 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.918146 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.918147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.918147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.918213 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:25.918487 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:25.918488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:25.918488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:25.918657 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:25.918791 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.918956 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.919036 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 21:00:25.921420 9984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 21:00:25.923213 9984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 21:00:25.930840 10652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.926163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:38589 (local address 127.0.0.1:46754)
0808 21:00:25.926528 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:25.926540 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:25.926596 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:25.926973 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:25.926974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:25.926976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:25.927055 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.927057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.928163 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.928164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.929522 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.929525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.930573 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.930574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.930640 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.930647 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 21:00:25.930703 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 21:00:25.930736 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":42,"thread_start_us":76,"threads_started":1}
I20260808 21:00:25.930888 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.926260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.15.232.131:38589)
0808 21:00:25.926800 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.926804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.926814 (+ 10us) server_negotiation.cc:407] Connection header received
0808 21:00:25.926850 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.926853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.926868 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.927141 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:25.927220 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.928045 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.929623 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.930460 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.930549 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.930607 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.930737 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.930790 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":96,"threads_started":1}
I20260808 21:00:25.930950 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.926577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52266 (local address 127.15.232.129:34493)
0808 21:00:25.926736 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.926741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.926754 (+ 13us) server_negotiation.cc:407] Connection header received
0808 21:00:25.926810 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.926812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.926830 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.926917 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:25.927129 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.928392 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.929697 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.930210 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.930327 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.930397 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.930785 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.930846 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260808 21:00:25.931214 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.926163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:38707 (local address 127.0.0.1:54942)
0808 21:00:25.926359 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:25.926407 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:25.926491 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:25.926775 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:25.926778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:25.926780 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:25.926856 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.926858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.928442 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.928443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.929523 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.929526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.930653 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.930661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.930752 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.930757 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 21:00:25.930814 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 21:00:25.930862 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":116,"threads_started":1}
I20260808 21:00:25.930738 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.926394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:34493 (local address 127.0.0.1:52266)
0808 21:00:25.926636 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:25.926646 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:25.926692 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:25.926976 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:25.926977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:25.926979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:25.927033 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.927035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.928509 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.928510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.929629 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.929631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.930333 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.930334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.930428 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.930452 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 21:00:25.930561 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 21:00:25.930622 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260808 21:00:25.931567 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.926281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54942 (local address 127.15.232.130:38707)
0808 21:00:25.926511 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.926516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.926529 (+ 13us) server_negotiation.cc:407] Connection header received
0808 21:00:25.926581 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.926583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.926601 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.926690 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:25.926959 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.928297 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.929664 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.930476 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.930549 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.930607 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.931368 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.931473 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":94,"threads_started":1}
I20260808 21:00:25.932024 10532 tablet_service.cc:1467] Processing CreateTablet for tablet 88ce01eb26db45ffa63342300e866149 (DEFAULT_TABLE table=test-table [id=40eba9d161cf4fc0b0538640b1b1430f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:25.932186 10153 tablet_service.cc:1467] Processing CreateTablet for tablet 88ce01eb26db45ffa63342300e866149 (DEFAULT_TABLE table=test-table [id=40eba9d161cf4fc0b0538640b1b1430f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:25.932190 10353 tablet_service.cc:1467] Processing CreateTablet for tablet 88ce01eb26db45ffa63342300e866149 (DEFAULT_TABLE table=test-table [id=40eba9d161cf4fc0b0538640b1b1430f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:25.932495 10532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88ce01eb26db45ffa63342300e866149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:25.932562 10153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88ce01eb26db45ffa63342300e866149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:25.932562 10353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88ce01eb26db45ffa63342300e866149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:25.935503 10656 tablet_bootstrap.cc:492] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: Bootstrap starting.
I20260808 21:00:25.936146 10657 tablet_bootstrap.cc:492] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326: Bootstrap starting.
I20260808 21:00:25.936419 10658 tablet_bootstrap.cc:492] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad: Bootstrap starting.
I20260808 21:00:25.936671 10656 tablet_bootstrap.cc:654] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:25.936862 10657 tablet_bootstrap.cc:654] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:25.937112 10658 tablet_bootstrap.cc:654] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:25.937328 10656 log.cc:824] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:25.937563 10657 log.cc:824] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:25.937736 10658 log.cc:824] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:25.938283 10656 tablet_bootstrap.cc:492] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: No bootstrap required, opened a new log
I20260808 21:00:25.938352 10656 ts_tablet_manager.cc:1397] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 21:00:25.938510 10658 tablet_bootstrap.cc:492] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad: No bootstrap required, opened a new log
I20260808 21:00:25.938596 10658 ts_tablet_manager.cc:1397] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:25.940212 10657 tablet_bootstrap.cc:492] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326: No bootstrap required, opened a new log
I20260808 21:00:25.940297 10657 ts_tablet_manager.cc:1397] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 21:00:25.940266 10658 raft_consensus.cc:348] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } }
I20260808 21:00:25.940480 10658 raft_consensus.cc:374] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:25.940529 10658 raft_consensus.cc:729] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d2d93029ab6469a9ca4c6692a6aa3ad, State: Initialized, Role: FOLLOWER
I20260808 21:00:25.940575 10656 raft_consensus.cc:348] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } }
I20260808 21:00:25.940692 10658 consensus_queue.cc:260] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } }
I20260808 21:00:25.940791 10656 raft_consensus.cc:374] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:25.940842 10656 raft_consensus.cc:729] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07cb4d5fa55849c4901f86e4130b081e, State: Initialized, Role: FOLLOWER
I20260808 21:00:25.940956 10656 consensus_queue.cc:260] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } }
I20260808 21:00:25.940997 10658 ts_tablet_manager.cc:1428] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 21:00:25.941172 10443 heartbeater.cc:503] Master 127.15.232.190:32951 was elected leader, sending a full tablet report...
I20260808 21:00:25.941283 10656 ts_tablet_manager.cc:1428] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 21:00:25.941334 10633 heartbeater.cc:503] Master 127.15.232.190:32951 was elected leader, sending a full tablet report...
I20260808 21:00:25.942495 10657 raft_consensus.cc:348] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } }
I20260808 21:00:25.942728 10657 raft_consensus.cc:374] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:25.942782 10657 raft_consensus.cc:729] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 585059f6d61f4a8eb987b35871326326, State: Initialized, Role: FOLLOWER
I20260808 21:00:25.942903 10657 consensus_queue.cc:260] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } }
I20260808 21:00:25.943194 10657 ts_tablet_manager.cc:1428] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 21:00:25.943245 10255 heartbeater.cc:503] Master 127.15.232.190:32951 was elected leader, sending a full tablet report...
I20260808 21:00:25.956192 10662 raft_consensus.cc:482] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:25.956365 10662 raft_consensus.cc:504] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } }
I20260808 21:00:25.956862 10662 leader_election.cc:288] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07cb4d5fa55849c4901f86e4130b081e (127.15.232.131:38589), 585059f6d61f4a8eb987b35871326326 (127.15.232.129:34493)
W20260808 21:00:25.957590 10444 tablet.cc:2369] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 21:00:25.961342 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.957121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:38589 (local address 127.15.232.130:45825)
0808 21:00:25.957205 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:25.957221 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:25.957292 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:25.957599 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:25.957600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:25.957602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:25.957816 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.957819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.958842 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.958843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.960126 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.960129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.961006 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.961007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.961180 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.961185 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 21:00:25.961240 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 21:00:25.961274 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":60}
I20260808 21:00:25.961356 10666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.957239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:34493 (local address 127.15.232.130:43861)
0808 21:00:25.957506 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:25.957519 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:25.957619 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:25.957942 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:25.957943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:25.957945 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:25.958034 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.958036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.958852 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.958853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.959904 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:25.959907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.960761 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.960763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:25.961088 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.961097 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 21:00:25.961180 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 21:00:25.961248 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":190,"threads_started":1}
I20260808 21:00:25.961400 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.957354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:43861 (local address 127.15.232.129:34493)
0808 21:00:25.957409 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.957411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.957586 (+ 175us) server_negotiation.cc:407] Connection header received
0808 21:00:25.957787 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.957789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.957800 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.957871 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:25.958129 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.958789 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.960000 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.960545 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.960627 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.960637 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.961261 (+ 624us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.961321 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 21:00:25.961421 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.957333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:45825 (local address 127.15.232.131:38589)
0808 21:00:25.957401 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.957403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.957413 (+ 10us) server_negotiation.cc:407] Connection header received
0808 21:00:25.957444 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.957445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.957455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.957540 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:25.957912 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.958696 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.960234 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.960782 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.960837 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.960842 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.961279 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.961326 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 21:00:25.962329 10184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ce01eb26db45ffa63342300e866149" candidate_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "585059f6d61f4a8eb987b35871326326" is_pre_election: true
I20260808 21:00:25.962500 10184 raft_consensus.cc:2454] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d2d93029ab6469a9ca4c6692a6aa3ad in term 0.
I20260808 21:00:25.962401 10566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ce01eb26db45ffa63342300e866149" candidate_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cb4d5fa55849c4901f86e4130b081e" is_pre_election: true
I20260808 21:00:25.962662 10566 raft_consensus.cc:2454] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d2d93029ab6469a9ca4c6692a6aa3ad in term 0.
I20260808 21:00:25.962800 10290 leader_election.cc:302] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d2d93029ab6469a9ca4c6692a6aa3ad, 585059f6d61f4a8eb987b35871326326; no voters:
I20260808 21:00:25.962990 10662 raft_consensus.cc:2781] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 21:00:25.963088 10662 raft_consensus.cc:482] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 21:00:25.963125 10662 raft_consensus.cc:3037] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:25.964010 10662 raft_consensus.cc:504] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } }
I20260808 21:00:25.964177 10662 leader_election.cc:288] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [CANDIDATE]: Term 1 election: Requested vote from peers 07cb4d5fa55849c4901f86e4130b081e (127.15.232.131:38589), 585059f6d61f4a8eb987b35871326326 (127.15.232.129:34493)
I20260808 21:00:25.964412 10566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ce01eb26db45ffa63342300e866149" candidate_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07cb4d5fa55849c4901f86e4130b081e"
I20260808 21:00:25.964478 10184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ce01eb26db45ffa63342300e866149" candidate_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "585059f6d61f4a8eb987b35871326326"
I20260808 21:00:25.964591 10184 raft_consensus.cc:3037] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:25.964504 10566 raft_consensus.cc:3037] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:25.965385 10184 raft_consensus.cc:2454] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d2d93029ab6469a9ca4c6692a6aa3ad in term 1.
I20260808 21:00:25.965504 10566 raft_consensus.cc:2454] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d2d93029ab6469a9ca4c6692a6aa3ad in term 1.
I20260808 21:00:25.965617 10290 leader_election.cc:302] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d2d93029ab6469a9ca4c6692a6aa3ad, 585059f6d61f4a8eb987b35871326326; no voters:
I20260808 21:00:25.965767 10662 raft_consensus.cc:2781] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:25.965955 10662 raft_consensus.cc:686] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 1 LEADER]: Becoming Leader. State: Replica: 4d2d93029ab6469a9ca4c6692a6aa3ad, State: Running, Role: LEADER
I20260808 21:00:25.966074 10662 consensus_queue.cc:237] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } }
I20260808 21:00:25.967051 9983 catalog_manager.cc:5495] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d2d93029ab6469a9ca4c6692a6aa3ad (127.15.232.130). New cstate: current_term: 1 leader_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d2d93029ab6469a9ca4c6692a6aa3ad" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 38707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 } health_report { overall_health: UNKNOWN } } }
I20260808 21:00:25.983229 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:25.980253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54954 (local address 127.15.232.130:38707)
0808 21:00:25.980307 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:25.980310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:25.980322 (+ 12us) server_negotiation.cc:407] Connection header received
0808 21:00:25.980372 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:25.980373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:25.980398 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:25.980500 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:00:25.980832 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.981378 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.982333 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:25.982561 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:25.982660 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:25.982832 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:00:25.982946 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:00:25.983075 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:25.983110 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 21:00:25.983125 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 21:00:25.985957 10566 raft_consensus.cc:1261] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [term 1 FOLLOWER]: Refusing update from remote peer 4d2d93029ab6469a9ca4c6692a6aa3ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:25.985957 10184 raft_consensus.cc:1261] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [term 1 FOLLOWER]: Refusing update from remote peer 4d2d93029ab6469a9ca4c6692a6aa3ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:25.986356 10662 consensus_queue.cc:1035] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "07cb4d5fa55849c4901f86e4130b081e" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:25.986616 10668 consensus_queue.cc:1035] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "585059f6d61f4a8eb987b35871326326" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 34493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:25.989729 10674 mvcc.cc:205] Tried to move back new op lower bound from 7316368695234936832 to 7316368695157805056. Current Snapshot: MvccSnapshot[applied={T|T < 7316368695234936832}]
I20260808 21:00:25.989850 10672 mvcc.cc:205] Tried to move back new op lower bound from 7316368695234936832 to 7316368695157805056. Current Snapshot: MvccSnapshot[applied={T|T < 7316368695234936832}]
I20260808 21:00:25.992554 9983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:25.992667 9983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:25.993602 9983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d66db36ece3c49db9fee228310ae6142: Sending DeleteTablet for 3 replicas of tablet 88ce01eb26db45ffa63342300e866149
I20260808 21:00:25.994059 10153 tablet_service.cc:1514] Processing DeleteTablet for tablet 88ce01eb26db45ffa63342300e866149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:25 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:52266
I20260808 21:00:25.994097 10532 tablet_service.cc:1514] Processing DeleteTablet for tablet 88ce01eb26db45ffa63342300e866149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:25 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:46754
I20260808 21:00:25.994130 10353 tablet_service.cc:1514] Processing DeleteTablet for tablet 88ce01eb26db45ffa63342300e866149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:25 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:54942
I20260808 21:00:25.994421 10687 tablet_replica.cc:330] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326: stopping tablet replica
I20260808 21:00:25.994459 10689 tablet_replica.cc:330] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad: stopping tablet replica
I20260808 21:00:25.994560 10687 raft_consensus.cc:2229] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:25.994681 10687 raft_consensus.cc:2258] T 88ce01eb26db45ffa63342300e866149 P 585059f6d61f4a8eb987b35871326326 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:25.994699 10689 raft_consensus.cc:2229] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 1 LEADER]: Raft consensus shutting down.
I20260808 21:00:25.994807 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 10069
I20260808 21:00:25.994864 10688 tablet_replica.cc:330] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: stopping tablet replica
I20260808 21:00:25.994966 10688 raft_consensus.cc:2229] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:25.995072 10688 raft_consensus.cc:2258] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:25.994993 10689 raft_consensus.cc:2258] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:26.008539 10689 ts_tablet_manager.cc:1905] T 88ce01eb26db45ffa63342300e866149 P 4d2d93029ab6469a9ca4c6692a6aa3ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:26.008683 10688 ts_tablet_manager.cc:1905] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 21:00:26.009158 9968 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:34493 (error 108)
I20260808 21:00:26.009910 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 10263
I20260808 21:00:26.016515 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.016314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:34493 (local address 127.0.0.1:52280)
0808 21:00:26.016393 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.016452 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:34493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 21:00:26.016727 9968 catalog_manager.cc:4584] TS 585059f6d61f4a8eb987b35871326326 (127.15.232.129:34493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88ce01eb26db45ffa63342300e866149: Network error: Client connection negotiation failed: client connection to 127.15.232.129:34493: connect: Connection refused (error 111)
I20260808 21:00:26.021689 10688 ts_tablet_manager.cc:1918] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:26.021855 10688 log.cc:1196] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/88ce01eb26db45ffa63342300e866149
I20260808 21:00:26.022181 10688 ts_tablet_manager.cc:1939] T 88ce01eb26db45ffa63342300e866149 P 07cb4d5fa55849c4901f86e4130b081e: Deleting consensus metadata
I20260808 21:00:26.023104 9968 catalog_manager.cc:4841] TS 07cb4d5fa55849c4901f86e4130b081e (127.15.232.131:38589): tablet 88ce01eb26db45ffa63342300e866149 (table test-table [id=40eba9d161cf4fc0b0538640b1b1430f]) successfully deleted
I20260808 21:00:26.027667 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 10451
I20260808 21:00:26.027871 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.027633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:38707 (local address 127.0.0.1:54962)
0808 21:00:26.027722 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.027818 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.130:38707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 21:00:26.040177 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 9938
2026-08-08T21:00:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2306 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T21:00:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:26Z Disabled control of system clock
I20260808 21:00:26.079802 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:44243
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:44243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 21:00:26.167896 10694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:26.168138 10694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:26.168161 10694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:26.169312 10694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:26.169354 10694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:26.169368 10694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:26.169380 10694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:26.169394 10694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:26.170954 10694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:44243
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:44243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:26.171195 10694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:26.171473 10694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:26.177183 10703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:26.177394 10694 server_base.cc:1034] running on GCE node
W20260808 21:00:26.177163 10700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:26.177161 10701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:26.177994 10694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:26.178342 10694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:26.179548 10694 hybrid_clock.cc:648] HybridClock initialized: now 1786222826179465 us; error 110 us; skew 500 ppm
I20260808 21:00:26.181090 10694 webserver.cc:460] Webserver started at http://127.15.232.190:44123/ using document root <none> and password file <none>
I20260808 21:00:26.181540 10694 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:26.181638 10694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:26.181948 10694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:26.183219 10694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "283be80887f144fcafec785516486ec8"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "72a9ef4e8f1ddcfeec80ca1d73128ba3"
server_key_iv: "86a9176196c7893bdc7340dc55f3ee36"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.183691 10694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "283be80887f144fcafec785516486ec8"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "72a9ef4e8f1ddcfeec80ca1d73128ba3"
server_key_iv: "86a9176196c7893bdc7340dc55f3ee36"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.185523 10694 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 21:00:26.186502 10718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.186834 10694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:26.186980 10694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "283be80887f144fcafec785516486ec8"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "72a9ef4e8f1ddcfeec80ca1d73128ba3"
server_key_iv: "86a9176196c7893bdc7340dc55f3ee36"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.187078 10694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:26.213807 10694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:26.214229 10694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:26.214417 10694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:26.219795 10694 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:44243
I20260808 21:00:26.219862 10795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:44243 every 8 connection(s)
I20260808 21:00:26.220286 10694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:26.221407 10796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:26.223959 10796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8: Bootstrap starting.
I20260808 21:00:26.224611 10796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:26.224952 10796 log.cc:824] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:26.225747 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 10694
I20260808 21:00:26.225775 10796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8: No bootstrap required, opened a new log
I20260808 21:00:26.225852 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:26.226080 16290 external_mini_cluster.cc:1428] Setting key 5883c564a537f6d4c6aae0375938a189
I20260808 21:00:26.228075 10796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283be80887f144fcafec785516486ec8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 44243 } }
I20260808 21:00:26.228250 10796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:26.228318 10796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283be80887f144fcafec785516486ec8, State: Initialized, Role: FOLLOWER
I20260808 21:00:26.228479 10796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283be80887f144fcafec785516486ec8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 44243 } }
I20260808 21:00:26.228572 10796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:26.228610 10796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:26.228664 10796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:26.229305 10796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283be80887f144fcafec785516486ec8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 44243 } }
I20260808 21:00:26.229435 10796 leader_election.cc:302] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 283be80887f144fcafec785516486ec8; no voters:
I20260808 21:00:26.229678 10796 leader_election.cc:288] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:26.229724 10802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:26.229840 10802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [term 1 LEADER]: Becoming Leader. State: Replica: 283be80887f144fcafec785516486ec8, State: Running, Role: LEADER
I20260808 21:00:26.229981 10796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:26.230000 10802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283be80887f144fcafec785516486ec8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 44243 } }
I20260808 21:00:26.230403 10802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 283be80887f144fcafec785516486ec8. Latest consensus state: current_term: 1 leader_uuid: "283be80887f144fcafec785516486ec8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283be80887f144fcafec785516486ec8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 44243 } } }
I20260808 21:00:26.230695 10802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:26.230602 10804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "283be80887f144fcafec785516486ec8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "283be80887f144fcafec785516486ec8" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 44243 } } }
I20260808 21:00:26.231175 10804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:26.231182 10812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:26.231956 10812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:26.233758 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.226755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59262 (local address 127.15.232.190:44243)
0808 21:00:26.227002 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.227010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.227052 (+ 42us) server_negotiation.cc:407] Connection header received
0808 21:00:26.228061 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.228070 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.228139 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.228313 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:26.228570 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.230226 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.231404 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.232044 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.232151 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.232979 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:26.232984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:26.232985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:26.233211 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:26.233258 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:26.233270 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:26.233357 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.233436 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.233530 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260808 21:00:26.234524 10812 catalog_manager.cc:1347] Generated new cluster ID: 445d04b71570442497596cb159f61cd4
I20260808 21:00:26.234584 10812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:26.248543 10812 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:26.249370 10812 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:26.261492 10812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8: Generated new TSK 0
I20260808 21:00:26.261752 10812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:26.270414 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:44243
--builtin_ntp_servers=127.15.232.148:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 21:00:26.360010 10830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:26.360293 10830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:26.360337 10830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:26.361652 10830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:26.361732 10830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:26.361778 10830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 21:00:26.363301 10830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:44243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:26.363636 10830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:26.363981 10830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:26.369470 10843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:26.369482 10841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:26.369482 10840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:26.369757 10830 server_base.cc:1034] running on GCE node
I20260808 21:00:26.370025 10830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:26.370321 10830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:26.371522 10830 hybrid_clock.cc:648] HybridClock initialized: now 1786222826371491 us; error 49 us; skew 500 ppm
I20260808 21:00:26.373167 10830 webserver.cc:460] Webserver started at http://127.15.232.129:45939/ using document root <none> and password file <none>
I20260808 21:00:26.373469 10830 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:26.373548 10830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:26.373718 10830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:26.374874 10830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "aa5d2ef4d5d6419fa9e487b0309686bd"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "f234cfcaad4b6c67960d04f195577193"
server_key_iv: "501acbdd010f01103af4cf9b2e4f2be1"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.375272 10830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "aa5d2ef4d5d6419fa9e487b0309686bd"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "f234cfcaad4b6c67960d04f195577193"
server_key_iv: "501acbdd010f01103af4cf9b2e4f2be1"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.377010 10830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:26.377946 10853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.378176 10830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:26.378290 10830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "aa5d2ef4d5d6419fa9e487b0309686bd"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "f234cfcaad4b6c67960d04f195577193"
server_key_iv: "501acbdd010f01103af4cf9b2e4f2be1"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.378386 10830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:26.386363 10830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:26.386715 10830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:26.386878 10830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:26.387133 10830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:26.387570 10830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:26.387609 10830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.387632 10830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:26.387645 10830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.394086 10830 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:39907
I20260808 21:00:26.394184 11012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:39907 every 8 connection(s)
I20260808 21:00:26.394589 10830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 21:00:26.397529 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 10830
I20260808 21:00:26.397632 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 21:00:26.397846 16290 external_mini_cluster.cc:1428] Setting key d81ee5e08761464dbc272edbbf7d5bb9
I20260808 21:00:26.398896 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:44243
--builtin_ntp_servers=127.15.232.148:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 21:00:26.401289 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.395512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:55729 (local address 127.15.232.190:44243)
0808 21:00:26.395567 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.395568 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.395852 (+ 284us) server_negotiation.cc:407] Connection header received
0808 21:00:26.396437 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.396439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.396453 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.396523 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:26.397351 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.398019 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.399327 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.399677 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.399760 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.400904 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:26.400905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:26.400906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:26.400915 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:26.400934 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:26.400939 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:26.401025 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.401155 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.401193 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 21:00:26.401582 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.395516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:44243 (local address 127.15.232.129:55729)
0808 21:00:26.395738 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.395774 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:26.396300 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:26.396693 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:26.396697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:26.396705 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:26.397244 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.397249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.398127 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.398128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.399228 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.399231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.399807 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.399808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.399962 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.400699 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 21:00:26.400700 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:26.400743 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 21:00:26.400772 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:26.401025 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:26.401033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:26.401037 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:26.401155 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 21:00:26.401275 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260808 21:00:26.402197 11014 heartbeater.cc:347] Connected to a master server at 127.15.232.190:44243
I20260808 21:00:26.402315 11014 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:26.402529 11014 heartbeater.cc:511] Master 127.15.232.190:44243 requested a full tablet report, sending...
I20260808 21:00:26.403249 10745 ts_manager.cc:194] Registered new tserver with Master: aa5d2ef4d5d6419fa9e487b0309686bd (127.15.232.129:39907)
I20260808 21:00:26.403995 10745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.129:55729
W20260808 21:00:26.486887 11019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:26.487126 11019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:26.487149 11019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:26.488405 11019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:26.488451 11019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:26.488482 11019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 21:00:26.489984 11019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:44243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:26.490264 11019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:26.490695 11019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:26.496177 11027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:26.496229 11026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:26.496259 11019 server_base.cc:1034] running on GCE node
W20260808 21:00:26.496193 11030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:26.496737 11019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:26.497048 11019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:26.498252 11019 hybrid_clock.cc:648] HybridClock initialized: now 1786222826498225 us; error 52 us; skew 500 ppm
I20260808 21:00:26.499841 11019 webserver.cc:460] Webserver started at http://127.15.232.130:34033/ using document root <none> and password file <none>
I20260808 21:00:26.500136 11019 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:26.500214 11019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:26.500373 11019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:26.501498 11019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "fdc0ef9a052b42c7965a38d66b190130"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "7d3353d42d332307a4a50c0fed9977f4"
server_key_iv: "f8841d2586627fa35673783767a108ee"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.501869 11019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "fdc0ef9a052b42c7965a38d66b190130"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "7d3353d42d332307a4a50c0fed9977f4"
server_key_iv: "f8841d2586627fa35673783767a108ee"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.503353 11019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 21:00:26.504343 11040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.504662 11019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:26.504815 11019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "fdc0ef9a052b42c7965a38d66b190130"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "7d3353d42d332307a4a50c0fed9977f4"
server_key_iv: "f8841d2586627fa35673783767a108ee"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.504917 11019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:26.525381 11019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:26.525779 11019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:26.525969 11019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:26.526235 11019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:26.526683 11019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:26.526744 11019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.526799 11019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:26.526842 11019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.533335 11019 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:42661
I20260808 21:00:26.533442 11198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:42661 every 8 connection(s)
I20260808 21:00:26.533802 11019 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 21:00:26.535924 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 11019
I20260808 21:00:26.536062 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 21:00:26.536391 16290 external_mini_cluster.cc:1428] Setting key 571979fe0719092d8e8f2625c7b35dde
I20260808 21:00:26.537533 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:44243
--builtin_ntp_servers=127.15.232.148:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 21:00:26.540303 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.534709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:58207 (local address 127.15.232.190:44243)
0808 21:00:26.534794 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.534796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.535048 (+ 252us) server_negotiation.cc:407] Connection header received
0808 21:00:26.535730 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.535732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.535744 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.535810 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:26.537084 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.537663 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.538736 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.538949 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.539033 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.539983 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:26.539985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:26.539985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:26.539991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:26.540007 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:26.540011 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:26.540085 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.540185 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.540219 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 21:00:26.540540 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.534685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:44243 (local address 127.15.232.130:58207)
0808 21:00:26.534907 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.534946 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:26.535577 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:26.535903 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:26.535908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:26.535920 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:26.536776 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.536784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.537787 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.537788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.538630 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.538633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.539046 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.539047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.539186 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.539808 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 21:00:26.539808 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:26.539840 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 21:00:26.539865 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:26.540085 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:26.540092 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:26.540095 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 21:00:26.540186 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 21:00:26.540302 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":108,"threads_started":1}
I20260808 21:00:26.541098 11199 heartbeater.cc:347] Connected to a master server at 127.15.232.190:44243
I20260808 21:00:26.541214 11199 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:26.541404 11199 heartbeater.cc:511] Master 127.15.232.190:44243 requested a full tablet report, sending...
I20260808 21:00:26.541941 10745 ts_manager.cc:194] Registered new tserver with Master: fdc0ef9a052b42c7965a38d66b190130 (127.15.232.130:42661)
I20260808 21:00:26.542461 10745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.130:58207
W20260808 21:00:26.628778 11202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:26.629014 11202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:26.629037 11202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:26.630342 11202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:26.630497 11202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:26.630558 11202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 21:00:26.632198 11202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:44243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:26.632495 11202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:26.632822 11202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:26.638374 11210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:26.638578 11213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:26.638681 11202 server_base.cc:1034] running on GCE node
W20260808 21:00:26.638779 11211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:26.639042 11202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:26.639278 11202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:26.640434 11202 hybrid_clock.cc:648] HybridClock initialized: now 1786222826640409 us; error 44 us; skew 500 ppm
I20260808 21:00:26.642184 11202 webserver.cc:460] Webserver started at http://127.15.232.131:45445/ using document root <none> and password file <none>
I20260808 21:00:26.642489 11202 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:26.642546 11202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:26.642695 11202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:26.643958 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "d2825930908f4bf3971b5b2c37e6feff"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "bd4837b41c4f1efc00d26a9a59ad6c3a"
server_key_iv: "0f0f635b4110fad409e8a6dbb6c6a251"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.644358 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "d2825930908f4bf3971b5b2c37e6feff"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "bd4837b41c4f1efc00d26a9a59ad6c3a"
server_key_iv: "0f0f635b4110fad409e8a6dbb6c6a251"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.645891 11202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:00:26.647001 11222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.647264 11202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:26.647398 11202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "d2825930908f4bf3971b5b2c37e6feff"
format_stamp: "Formatted at 2026-08-08 21:00:26 on dist-test-slave-c12x"
server_key: "bd4837b41c4f1efc00d26a9a59ad6c3a"
server_key_iv: "0f0f635b4110fad409e8a6dbb6c6a251"
server_key_version: "encryptionkey@0"
I20260808 21:00:26.647541 11202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:26.661432 11202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:26.661826 11202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:26.662022 11202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:26.662315 11202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:26.662716 11202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:26.662781 11202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.662822 11202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:26.662870 11202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:26.669510 11202 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:36743
I20260808 21:00:26.669569 11368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:36743 every 8 connection(s)
I20260808 21:00:26.669982 11202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 21:00:26.674925 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 11202
I20260808 21:00:26.675082 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 21:00:26.675338 16290 external_mini_cluster.cc:1428] Setting key 97621d9e366534d62af840b073874610
I20260808 21:00:26.676549 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.670944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:37887 (local address 127.15.232.190:44243)
0808 21:00:26.671144 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.671147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.671357 (+ 210us) server_negotiation.cc:407] Connection header received
0808 21:00:26.672148 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.672150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.672165 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.672229 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:26.673031 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.673603 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.674777 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.674988 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.675123 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.676251 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:26.676253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:26.676253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:26.676259 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:26.676275 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:26.676279 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:26.676353 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.676432 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.676467 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260808 21:00:26.676781 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.670834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:44243 (local address 127.15.232.131:37887)
0808 21:00:26.671211 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.671258 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:26.672004 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:26.672310 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:26.672315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:26.672327 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:26.672918 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.672924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.673719 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.673720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.674665 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.674669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.675145 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.675146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.675319 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.676078 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 21:00:26.676078 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:26.676111 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 21:00:26.676136 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:26.676353 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:26.676360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:26.676364 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:26.676435 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 21:00:26.676551 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":226,"threads_started":1}
I20260808 21:00:26.677246 11371 heartbeater.cc:347] Connected to a master server at 127.15.232.190:44243
I20260808 21:00:26.677436 11371 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:26.677670 11371 heartbeater.cc:511] Master 127.15.232.190:44243 requested a full tablet report, sending...
I20260808 21:00:26.678139 10745 ts_manager.cc:194] Registered new tserver with Master: d2825930908f4bf3971b5b2c37e6feff (127.15.232.131:36743)
I20260808 21:00:26.678578 10745 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.131:37887
I20260808 21:00:26.687819 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 21:00:26.693423 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.689647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.15.232.190:44243)
0808 21:00:26.689722 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.689724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.689835 (+ 111us) server_negotiation.cc:407] Connection header received
0808 21:00:26.689926 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.689927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.689951 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.690026 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:26.690280 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.690859 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.691860 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.692079 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.692185 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.693069 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:26.693070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:26.693071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:26.693076 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:26.693093 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:26.693098 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:26.693219 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.693297 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.693311 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 21:00:26.695909 10745 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 21:00:26.697638 10745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 21:00:26.705523 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.701007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.15.232.131:36743)
0808 21:00:26.701527 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.701531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.701543 (+ 12us) server_negotiation.cc:407] Connection header received
0808 21:00:26.701585 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.701587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.701603 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.701867 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:26.701946 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.702725 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.704317 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.705023 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.705117 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.705222 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.705381 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.705432 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":107,"threads_started":1}
I20260808 21:00:26.705564 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.700798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:42661 (local address 127.0.0.1:45948)
0808 21:00:26.701060 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.701110 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:26.701195 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:26.701532 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:26.701535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:26.701536 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:26.701612 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.701618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.703172 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.703173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.704143 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.704146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.705131 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.705132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.705252 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.705268 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 21:00:26.705347 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 21:00:26.705423 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":25,"thread_start_us":71,"threads_started":1}
I20260808 21:00:26.705760 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.701118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45948 (local address 127.15.232.130:42661)
0808 21:00:26.701312 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.701318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.701330 (+ 12us) server_negotiation.cc:407] Connection header received
0808 21:00:26.701382 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.701385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.701402 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.701520 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:26.701812 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.703055 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.704231 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.705020 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.705479 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.705578 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.705607 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.705653 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
I20260808 21:00:26.705826 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.700826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:36743 (local address 127.0.0.1:49754)
0808 21:00:26.701335 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.701344 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:26.701386 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:26.701719 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:26.701720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:26.701722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:26.701787 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.701789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.702827 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.702829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.704214 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.704217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.705149 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.705151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.705247 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.705268 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 21:00:26.705352 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 21:00:26.705447 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":39,"thread_start_us":139,"threads_started":1}
I20260808 21:00:26.706067 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.701081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34606 (local address 127.15.232.129:39907)
0808 21:00:26.701282 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.701287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.701661 (+ 374us) server_negotiation.cc:407] Connection header received
0808 21:00:26.701710 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.701713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.701729 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.701812 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:26.702018 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.703145 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.704532 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.705071 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.705182 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.705285 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.705613 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.705972 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
I20260808 21:00:26.706530 11283 tablet_service.cc:1467] Processing CreateTablet for tablet 5481f26cc2124ccaa650c8b0f403b556 (DEFAULT_TABLE table=test-table [id=0279cc515a794b27a52622f038b1918e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:26.706600 11110 tablet_service.cc:1467] Processing CreateTablet for tablet 5481f26cc2124ccaa650c8b0f403b556 (DEFAULT_TABLE table=test-table [id=0279cc515a794b27a52622f038b1918e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:26.706888 11283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5481f26cc2124ccaa650c8b0f403b556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:26.707038 11110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5481f26cc2124ccaa650c8b0f403b556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:26.705567 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.700814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:39907 (local address 127.0.0.1:34606)
0808 21:00:26.701304 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.701316 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:26.701402 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:26.701888 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:26.701889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:26.701890 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:26.701927 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.701928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.703241 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.703242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.704424 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.704426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.705323 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.705324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.705394 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.705399 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 21:00:26.705463 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 21:00:26.705501 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":66,"threads_started":1}
I20260808 21:00:26.708470 10928 tablet_service.cc:1467] Processing CreateTablet for tablet 5481f26cc2124ccaa650c8b0f403b556 (DEFAULT_TABLE table=test-table [id=0279cc515a794b27a52622f038b1918e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:26.708891 10928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5481f26cc2124ccaa650c8b0f403b556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:26.710282 11393 tablet_bootstrap.cc:492] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: Bootstrap starting.
I20260808 21:00:26.710947 11393 tablet_bootstrap.cc:654] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:26.710929 11396 tablet_bootstrap.cc:492] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: Bootstrap starting.
I20260808 21:00:26.711201 11393 log.cc:824] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:26.711609 11396 tablet_bootstrap.cc:654] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:26.711943 11396 log.cc:824] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:26.712077 11393 tablet_bootstrap.cc:492] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: No bootstrap required, opened a new log
I20260808 21:00:26.712167 11393 ts_tablet_manager.cc:1397] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:26.712630 11399 tablet_bootstrap.cc:492] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd: Bootstrap starting.
I20260808 21:00:26.712790 11396 tablet_bootstrap.cc:492] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: No bootstrap required, opened a new log
I20260808 21:00:26.712880 11396 ts_tablet_manager.cc:1397] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 21:00:26.713305 11399 tablet_bootstrap.cc:654] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:26.713629 11399 log.cc:824] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:26.714334 11399 tablet_bootstrap.cc:492] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd: No bootstrap required, opened a new log
I20260808 21:00:26.714443 11399 ts_tablet_manager.cc:1397] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 21:00:26.715263 11393 raft_consensus.cc:348] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } }
I20260808 21:00:26.715631 11393 raft_consensus.cc:374] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:26.715739 11393 raft_consensus.cc:729] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2825930908f4bf3971b5b2c37e6feff, State: Initialized, Role: FOLLOWER
I20260808 21:00:26.715950 11393 consensus_queue.cc:260] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } }
I20260808 21:00:26.715898 11396 raft_consensus.cc:348] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } }
I20260808 21:00:26.716071 11396 raft_consensus.cc:374] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:26.716157 11396 raft_consensus.cc:729] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc0ef9a052b42c7965a38d66b190130, State: Initialized, Role: FOLLOWER
I20260808 21:00:26.716293 11393 ts_tablet_manager.cc:1428] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 21:00:26.716296 11396 consensus_queue.cc:260] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } }
I20260808 21:00:26.716445 11371 heartbeater.cc:503] Master 127.15.232.190:44243 was elected leader, sending a full tablet report...
I20260808 21:00:26.716634 11396 ts_tablet_manager.cc:1428] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 21:00:26.716656 11199 heartbeater.cc:503] Master 127.15.232.190:44243 was elected leader, sending a full tablet report...
I20260808 21:00:26.717270 11399 raft_consensus.cc:348] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } }
I20260808 21:00:26.717726 11399 raft_consensus.cc:374] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:26.717828 11399 raft_consensus.cc:729] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa5d2ef4d5d6419fa9e487b0309686bd, State: Initialized, Role: FOLLOWER
I20260808 21:00:26.718081 11399 consensus_queue.cc:260] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } }
I20260808 21:00:26.718449 11399 ts_tablet_manager.cc:1428] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 21:00:26.718540 11014 heartbeater.cc:503] Master 127.15.232.190:44243 was elected leader, sending a full tablet report...
W20260808 21:00:26.784595 11200 tablet.cc:2369] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 21:00:26.895592 11015 tablet.cc:2369] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 21:00:26.920662 11373 tablet.cc:2369] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 21:00:26.921160 11404 raft_consensus.cc:482] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:26.921306 11404 raft_consensus.cc:504] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } }
I20260808 21:00:26.921816 11404 leader_election.cc:288] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdc0ef9a052b42c7965a38d66b190130 (127.15.232.130:42661), d2825930908f4bf3971b5b2c37e6feff (127.15.232.131:36743)
I20260808 21:00:26.925244 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.922181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:44503 (local address 127.15.232.130:42661)
0808 21:00:26.922235 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.922237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.922438 (+ 201us) server_negotiation.cc:407] Connection header received
0808 21:00:26.922507 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.922508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.922521 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.922589 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:26.922961 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.923453 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.924515 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.924874 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.924969 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.924977 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.925121 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.925163 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 21:00:26.925267 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.922190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:52807 (local address 127.15.232.131:36743)
0808 21:00:26.922247 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.922250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.922379 (+ 129us) server_negotiation.cc:407] Connection header received
0808 21:00:26.922420 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.922422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.922435 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.922502 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:26.922791 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.923381 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.924488 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.924868 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.924963 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.924971 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.925160 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.925206 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 21:00:26.925268 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.922100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:42661 (local address 127.15.232.129:44503)
0808 21:00:26.922284 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.922299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:26.922429 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:26.922613 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:26.922614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:26.922616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:26.922816 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.922819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.923623 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.923624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.924376 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.924378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.924967 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.924968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.925047 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.925051 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:26.925094 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 21:00:26.925160 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":78,"threads_started":1}
I20260808 21:00:26.925273 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.922116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:36743 (local address 127.15.232.129:52807)
0808 21:00:26.922288 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.922305 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:26.922368 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:26.922512 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:26.922513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:26.922515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:26.922605 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.922609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.923505 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.923507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.924359 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:26.924362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.924963 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.924964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:26.925054 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.925059 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 21:00:26.925134 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 21:00:26.925176 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260808 21:00:26.925791 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5481f26cc2124ccaa650c8b0f403b556" candidate_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdc0ef9a052b42c7965a38d66b190130" is_pre_election: true
I20260808 21:00:26.925985 11135 raft_consensus.cc:2454] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa5d2ef4d5d6419fa9e487b0309686bd in term 0.
I20260808 21:00:26.925952 11305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5481f26cc2124ccaa650c8b0f403b556" candidate_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2825930908f4bf3971b5b2c37e6feff" is_pre_election: true
I20260808 21:00:26.926141 11305 raft_consensus.cc:2454] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa5d2ef4d5d6419fa9e487b0309686bd in term 0.
I20260808 21:00:26.926298 10857 leader_election.cc:302] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa5d2ef4d5d6419fa9e487b0309686bd, fdc0ef9a052b42c7965a38d66b190130; no voters:
I20260808 21:00:26.926458 11404 raft_consensus.cc:2781] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 21:00:26.926532 11404 raft_consensus.cc:482] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 21:00:26.926566 11404 raft_consensus.cc:3037] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:26.927342 11404 raft_consensus.cc:504] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } }
I20260808 21:00:26.927537 11404 leader_election.cc:288] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [CANDIDATE]: Term 1 election: Requested vote from peers fdc0ef9a052b42c7965a38d66b190130 (127.15.232.130:42661), d2825930908f4bf3971b5b2c37e6feff (127.15.232.131:36743)
I20260808 21:00:26.927707 11305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5481f26cc2124ccaa650c8b0f403b556" candidate_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2825930908f4bf3971b5b2c37e6feff"
I20260808 21:00:26.927744 11135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5481f26cc2124ccaa650c8b0f403b556" candidate_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdc0ef9a052b42c7965a38d66b190130"
I20260808 21:00:26.927800 11305 raft_consensus.cc:3037] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:26.927814 11135 raft_consensus.cc:3037] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:26.928810 11305 raft_consensus.cc:2454] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa5d2ef4d5d6419fa9e487b0309686bd in term 1.
I20260808 21:00:26.928810 11135 raft_consensus.cc:2454] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa5d2ef4d5d6419fa9e487b0309686bd in term 1.
I20260808 21:00:26.929033 10854 leader_election.cc:302] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa5d2ef4d5d6419fa9e487b0309686bd, d2825930908f4bf3971b5b2c37e6feff; no voters:
I20260808 21:00:26.929189 11404 raft_consensus.cc:2781] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:26.929388 11404 raft_consensus.cc:686] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 1 LEADER]: Becoming Leader. State: Replica: aa5d2ef4d5d6419fa9e487b0309686bd, State: Running, Role: LEADER
I20260808 21:00:26.929507 11404 consensus_queue.cc:237] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } }
I20260808 21:00:26.930518 10745 catalog_manager.cc:5495] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd reported cstate change: term changed from 0 to 1, leader changed from <none> to aa5d2ef4d5d6419fa9e487b0309686bd (127.15.232.129). New cstate: current_term: 1 leader_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa5d2ef4d5d6419fa9e487b0309686bd" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 39907 } health_report { overall_health: HEALTHY } } }
I20260808 21:00:26.937197 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.934449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.15.232.129:39907)
0808 21:00:26.934492 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:26.934495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:26.934503 (+ 8us) server_negotiation.cc:407] Connection header received
0808 21:00:26.934547 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:26.934548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:26.934567 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:26.934641 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:00:26.934751 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.935265 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.936276 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:26.936520 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:26.936605 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:26.936774 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:00:26.936901 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:00:26.936996 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:26.937116 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 21:00:26.937128 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 21:00:26.939402 11135 raft_consensus.cc:1261] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [term 1 FOLLOWER]: Refusing update from remote peer aa5d2ef4d5d6419fa9e487b0309686bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:26.939411 11305 raft_consensus.cc:1261] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [term 1 FOLLOWER]: Refusing update from remote peer aa5d2ef4d5d6419fa9e487b0309686bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:26.939879 11409 consensus_queue.cc:1035] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdc0ef9a052b42c7965a38d66b190130" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:26.940059 11404 consensus_queue.cc:1035] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2825930908f4bf3971b5b2c37e6feff" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 36743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:26.943399 11413 mvcc.cc:205] Tried to move back new op lower bound from 7316368699140407296 to 7316368699104026624. Current Snapshot: MvccSnapshot[applied={T|T < 7316368699140407296}]
I20260808 21:00:26.944587 11414 mvcc.cc:205] Tried to move back new op lower bound from 7316368699140407296 to 7316368699104026624. Current Snapshot: MvccSnapshot[applied={T|T < 7316368699140407296}]
I20260808 21:00:26.946344 10744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:26.946475 10744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:26.947481 10744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 283be80887f144fcafec785516486ec8: Sending DeleteTablet for 3 replicas of tablet 5481f26cc2124ccaa650c8b0f403b556
I20260808 21:00:26.947973 11110 tablet_service.cc:1514] Processing DeleteTablet for tablet 5481f26cc2124ccaa650c8b0f403b556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:26 UTC) from {username='slave'} at 127.0.0.1:45948
I20260808 21:00:26.947973 11283 tablet_service.cc:1514] Processing DeleteTablet for tablet 5481f26cc2124ccaa650c8b0f403b556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:26 UTC) from {username='slave'} at 127.0.0.1:49754
I20260808 21:00:26.948323 10928 tablet_service.cc:1514] Processing DeleteTablet for tablet 5481f26cc2124ccaa650c8b0f403b556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:26 UTC) from {username='slave'} at 127.0.0.1:34606
I20260808 21:00:26.948609 11425 tablet_replica.cc:330] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd: stopping tablet replica
I20260808 21:00:26.948680 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 10830
I20260808 21:00:26.948817 11425 raft_consensus.cc:2229] T 5481f26cc2124ccaa650c8b0f403b556 P aa5d2ef4d5d6419fa9e487b0309686bd [term 1 LEADER]: Raft consensus shutting down.
I20260808 21:00:26.956272 11423 tablet_replica.cc:330] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: stopping tablet replica
I20260808 21:00:26.956452 11423 raft_consensus.cc:2229] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:26.956617 11423 raft_consensus.cc:2258] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:26.956876 11422 tablet_replica.cc:330] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: stopping tablet replica
I20260808 21:00:26.957048 11422 raft_consensus.cc:2229] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:26.957193 11422 raft_consensus.cc:2258] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:26.958204 11422 ts_tablet_manager.cc:1905] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:26.958197 11423 ts_tablet_manager.cc:1905] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:26.960479 11422 ts_tablet_manager.cc:1918] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:26.960561 11422 log.cc:1196] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/5481f26cc2124ccaa650c8b0f403b556
I20260808 21:00:26.960687 11423 ts_tablet_manager.cc:1918] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:26.960789 11422 ts_tablet_manager.cc:1939] T 5481f26cc2124ccaa650c8b0f403b556 P fdc0ef9a052b42c7965a38d66b190130: Deleting consensus metadata
I20260808 21:00:26.960881 11423 log.cc:1196] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/5481f26cc2124ccaa650c8b0f403b556
I20260808 21:00:26.961196 11423 ts_tablet_manager.cc:1939] T 5481f26cc2124ccaa650c8b0f403b556 P d2825930908f4bf3971b5b2c37e6feff: Deleting consensus metadata
I20260808 21:00:26.961469 10722 catalog_manager.cc:4841] TS fdc0ef9a052b42c7965a38d66b190130 (127.15.232.130:42661): tablet 5481f26cc2124ccaa650c8b0f403b556 (table test-table [id=0279cc515a794b27a52622f038b1918e]) successfully deleted
I20260808 21:00:26.961932 10719 catalog_manager.cc:4841] TS d2825930908f4bf3971b5b2c37e6feff (127.15.232.131:36743): tablet 5481f26cc2124ccaa650c8b0f403b556 (table test-table [id=0279cc515a794b27a52622f038b1918e]) successfully deleted
W20260808 21:00:26.964161 10725 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:39907 (error 108)
I20260808 21:00:26.964294 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 11019
I20260808 21:00:26.970167 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:26.969963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:39907 (local address 127.0.0.1:34616)
0808 21:00:26.970047 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:26.970112 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:39907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 21:00:26.970376 10725 catalog_manager.cc:4584] TS aa5d2ef4d5d6419fa9e487b0309686bd (127.15.232.129:39907): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5481f26cc2124ccaa650c8b0f403b556: Network error: Client connection negotiation failed: client connection to 127.15.232.129:39907: connect: Connection refused (error 111)
I20260808 21:00:26.976177 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 11202
I20260808 21:00:26.987959 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 10694
2026-08-08T21:00:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (950 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:27 dist-test-slave-c12x krb5kdc[11429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:27 dist-test-slave-c12x krb5kdc[11429](info): set up 2 sockets
Aug 08 21:00:27 dist-test-slave-c12x krb5kdc[11429](info): commencing operation
krb5kdc: starting...
W20260808 21:00:28.344969 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:28 dist-test-slave-c12x krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:00:28.495676 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:41747
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:41747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:28.589555 11444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:28.589830 11444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:28.589874 11444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:28.591073 11444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:28.591148 11444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:28.591174 11444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:28.591200 11444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:28.591233 11444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:28.592945 11444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:41747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:41747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:28.593247 11444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:28.593497 11444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:28.599092 11451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:28.599097 11452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:28.599177 11444 server_base.cc:1034] running on GCE node
W20260808 21:00:28.599195 11455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:28.599713 11444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:28.600068 11444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:28.601276 11444 hybrid_clock.cc:648] HybridClock initialized: now 1786222828601251 us; error 44 us; skew 500 ppm
Aug 08 21:00:28 dist-test-slave-c12x krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222828, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:28.604673 11444 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:28.605428 11444 webserver.cc:460] Webserver started at http://127.15.232.190:41685/ using document root <none> and password file <none>
I20260808 21:00:28.605691 11444 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:28.605772 11444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:28.605931 11444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:28.607120 11444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "caf88762d128447c983762220aaf549f"
format_stamp: "Formatted at 2026-08-08 21:00:28 on dist-test-slave-c12x"
server_key: "fd00eeb7b7126e214eed4f02eb6fd5a3"
server_key_iv: "795048317c353d2522232803203f6513"
server_key_version: "encryptionkey@0"
I20260808 21:00:28.607594 11444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "caf88762d128447c983762220aaf549f"
format_stamp: "Formatted at 2026-08-08 21:00:28 on dist-test-slave-c12x"
server_key: "fd00eeb7b7126e214eed4f02eb6fd5a3"
server_key_iv: "795048317c353d2522232803203f6513"
server_key_version: "encryptionkey@0"
I20260808 21:00:28.609174 11444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:28.610108 11462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:28.610359 11444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:28.610507 11444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "caf88762d128447c983762220aaf549f"
format_stamp: "Formatted at 2026-08-08 21:00:28 on dist-test-slave-c12x"
server_key: "fd00eeb7b7126e214eed4f02eb6fd5a3"
server_key_iv: "795048317c353d2522232803203f6513"
server_key_version: "encryptionkey@0"
I20260808 21:00:28.610595 11444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:28.631233 11444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:28.634054 11444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:28.634251 11444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:28.639358 11444 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:41747
I20260808 21:00:28.639400 11542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:41747 every 8 connection(s)
I20260808 21:00:28.639883 11444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:28.640885 11545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:28.643103 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 11444
I20260808 21:00:28.643229 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:28.643481 16290 external_mini_cluster.cc:1428] Setting key d72ac49d9d38440b64c76528c145ff89
I20260808 21:00:28.643810 11545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f: Bootstrap starting.
I20260808 21:00:28.644609 11545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:28.644938 11545 log.cc:824] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:28.645694 11545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f: No bootstrap required, opened a new log
I20260808 21:00:28.647789 11545 raft_consensus.cc:348] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf88762d128447c983762220aaf549f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41747 } }
I20260808 21:00:28.647997 11545 raft_consensus.cc:374] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:28.648047 11545 raft_consensus.cc:729] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf88762d128447c983762220aaf549f, State: Initialized, Role: FOLLOWER
I20260808 21:00:28.648188 11545 consensus_queue.cc:260] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf88762d128447c983762220aaf549f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41747 } }
I20260808 21:00:28.648293 11545 raft_consensus.cc:388] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:28.648339 11545 raft_consensus.cc:482] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:28.648398 11545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:28.649132 11545 raft_consensus.cc:504] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf88762d128447c983762220aaf549f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41747 } }
I20260808 21:00:28.649253 11545 leader_election.cc:302] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caf88762d128447c983762220aaf549f; no voters:
I20260808 21:00:28.649438 11545 leader_election.cc:288] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:28.649536 11550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:28.649811 11550 raft_consensus.cc:686] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [term 1 LEADER]: Becoming Leader. State: Replica: caf88762d128447c983762220aaf549f, State: Running, Role: LEADER
I20260808 21:00:28.649863 11545 sys_catalog.cc:564] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:28.650426 11550 consensus_queue.cc:237] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf88762d128447c983762220aaf549f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41747 } }
I20260808 21:00:28.651103 11556 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caf88762d128447c983762220aaf549f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf88762d128447c983762220aaf549f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41747 } } }
I20260808 21:00:28.651201 11556 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:28.651556 11572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:28.651919 11554 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [sys.catalog]: SysCatalogTable state changed. Reason: New leader caf88762d128447c983762220aaf549f. Latest consensus state: current_term: 1 leader_uuid: "caf88762d128447c983762220aaf549f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf88762d128447c983762220aaf549f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 41747 } } }
Aug 08 21:00:28 dist-test-slave-c12x krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:28.652180 11572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:28.652503 11554 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:28.654299 11572 catalog_manager.cc:1347] Generated new cluster ID: 2fa6013d18de431fa53e32c9cfdd5614
I20260808 21:00:28.654361 11572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:28.657768 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:28.644136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51920 (local address 127.15.232.190:41747)
0808 21:00:28.644358 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:28.644367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:28.644405 (+ 38us) server_negotiation.cc:407] Connection header received
0808 21:00:28.644976 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:28.644984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:28.645062 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:28.645227 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:28.645944 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:28.647370 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:28.648871 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:28.649563 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:28.649690 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:28.653200 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:28.653207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:28.653207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:28.653226 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:28.655573 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:28.656086 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:28.656090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:28.656090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:28.656214 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:28.656462 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:28.656463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:28.656464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:28.657019 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:28.657190 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:28.657351 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 21:00:28.657543 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":121,"threads_started":1}
I20260808 21:00:28.666869 11572 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:28.667537 11572 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:28.674113 11572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f: Generated new TSK 0
I20260808 21:00:28.674351 11572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.232.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
Entry for principal kudu/127.15.232.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab.
I20260808 21:00:28.709138 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41747
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:28.802225 11586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:28.802503 11586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:28.802552 11586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:28.803984 11586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:28.804064 11586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:28.804117 11586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 21:00:28.806034 11586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.129.keytab
--principal=kudu/127.15.232.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:28.806353 11586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:28.806623 11586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:28.812573 11596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:28.812551 11594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:28.812556 11598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:28.812961 11586 server_base.cc:1034] running on GCE node
I20260808 21:00:28.813241 11586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:28.813550 11586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:28.814731 11586 hybrid_clock.cc:648] HybridClock initialized: now 1786222828814709 us; error 50 us; skew 500 ppm
Aug 08 21:00:28 dist-test-slave-c12x krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222828, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:28.818023 11586 init.cc:377] Logged in from keytab as kudu/127.15.232.129@KRBTEST.COM (short username kudu)
I20260808 21:00:28.818562 11586 webserver.cc:460] Webserver started at http://127.15.232.129:40081/ using document root <none> and password file <none>
I20260808 21:00:28.818848 11586 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:28.818903 11586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:28.819051 11586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:28.820390 11586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "89b8c5fdc55e4ecd92d2148c00c31792"
format_stamp: "Formatted at 2026-08-08 21:00:28 on dist-test-slave-c12x"
server_key: "c02594f060293ea98a7afb25ee925aee"
server_key_iv: "aa6a483a2ee4218df3049a31c890ae02"
server_key_version: "encryptionkey@0"
I20260808 21:00:28.820976 11586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "89b8c5fdc55e4ecd92d2148c00c31792"
format_stamp: "Formatted at 2026-08-08 21:00:28 on dist-test-slave-c12x"
server_key: "c02594f060293ea98a7afb25ee925aee"
server_key_iv: "aa6a483a2ee4218df3049a31c890ae02"
server_key_version: "encryptionkey@0"
I20260808 21:00:28.822602 11586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 21:00:28.823997 11606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:28.824306 11586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:28.824431 11586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "89b8c5fdc55e4ecd92d2148c00c31792"
format_stamp: "Formatted at 2026-08-08 21:00:28 on dist-test-slave-c12x"
server_key: "c02594f060293ea98a7afb25ee925aee"
server_key_iv: "aa6a483a2ee4218df3049a31c890ae02"
server_key_version: "encryptionkey@0"
I20260808 21:00:28.824529 11586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:28.836200 11586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:28.838783 11586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:28.838973 11586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:28.839308 11586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:28.839920 11586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:28.840013 11586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:28.840070 11586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:28.840095 11586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:28.846869 11586 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:46507
I20260808 21:00:28.846918 11768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:46507 every 8 connection(s)
I20260808 21:00:28.847316 11586 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
Aug 08 21:00:28 dist-test-slave-c12x krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222828, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.129@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:28.856389 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 11586
I20260808 21:00:28.856526 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 21:00:28.856853 16290 external_mini_cluster.cc:1428] Setting key ea0fbeda4a031483a050d10fc4b870c4
I20260808 21:00:28.859543 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:28.848266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:43873 (local address 127.15.232.190:41747)
0808 21:00:28.848358 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:28.848366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:28.848546 (+ 180us) server_negotiation.cc:407] Connection header received
0808 21:00:28.849152 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:28.849153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:28.849164 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:28.849234 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:28.850454 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:28.851279 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:28.852581 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:28.852817 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:28.852886 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:28.855600 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:28.855602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:28.855602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:28.855613 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:28.857568 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:28.858187 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:28.858188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:28.858188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:28.858290 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:28.858677 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:28.858678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:28.858678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:28.858811 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:28.858882 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:28.859277 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 21:00:28.859364 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 21:00:28.859871 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:28.848226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41747 (local address 127.15.232.129:43873)
0808 21:00:28.848416 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:28.848466 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:28.849015 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:28.849295 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:28.849298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:28.850019 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:28.850318 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:28.850324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:28.851397 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:28.851398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:28.852478 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:28.852481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:28.852903 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:28.852904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:28.853028 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:28.853656 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:28.853666 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:28.855387 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:28.857658 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:28.857670 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:28.857673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:28.858093 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:28.858374 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:28.858374 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:28.858375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:28.858568 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:28.858893 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:28.858901 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:28.859109 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 21:00:28.859253 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 21:00:28.859514 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260808 21:00:28.860996 11769 heartbeater.cc:347] Connected to a master server at 127.15.232.190:41747
I20260808 21:00:28.861133 11769 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:28.861362 11769 heartbeater.cc:511] Master 127.15.232.190:41747 requested a full tablet report, sending...
I20260808 21:00:28.862004 11488 ts_manager.cc:194] Registered new tserver with Master: 89b8c5fdc55e4ecd92d2148c00c31792 (127.15.232.129:46507)
I20260808 21:00:28.863145 11488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.129@KRBTEST.COM'} at 127.15.232.129:43873
WARNING: no policy specified for kudu/127.15.232.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
Entry for principal kudu/127.15.232.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab.
I20260808 21:00:28.890431 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41747
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:28.991781 11779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:28.992023 11779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:28.992053 11779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:28.993237 11779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:28.993285 11779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:28.993314 11779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 21:00:28.994838 11779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.130.keytab
--principal=kudu/127.15.232.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:28.995059 11779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:28.995391 11779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:29.001185 11786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.001190 11789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.001185 11785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:29.001451 11779 server_base.cc:1034] running on GCE node
I20260808 21:00:29.001989 11779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:29.002288 11779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:29.003513 11779 hybrid_clock.cc:648] HybridClock initialized: now 1786222829003503 us; error 40 us; skew 500 ppm
Aug 08 21:00:29 dist-test-slave-c12x krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222829, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:29.006785 11779 init.cc:377] Logged in from keytab as kudu/127.15.232.130@KRBTEST.COM (short username kudu)
I20260808 21:00:29.007481 11779 webserver.cc:460] Webserver started at http://127.15.232.130:38719/ using document root <none> and password file <none>
I20260808 21:00:29.007875 11779 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:29.008003 11779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:29.008194 11779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:29.009931 11779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "64e2f38cbe444d3d975568fa868e52c8"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "5479a91126f9c313bb4a2e41490ab067"
server_key_iv: "9410c5e41d61bc6af232551445c09a33"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.010715 11779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "64e2f38cbe444d3d975568fa868e52c8"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "5479a91126f9c313bb4a2e41490ab067"
server_key_iv: "9410c5e41d61bc6af232551445c09a33"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.012713 11779 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260808 21:00:29.013742 11800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.013969 11779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:29.014127 11779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "64e2f38cbe444d3d975568fa868e52c8"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "5479a91126f9c313bb4a2e41490ab067"
server_key_iv: "9410c5e41d61bc6af232551445c09a33"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.014250 11779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:29.042464 11779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:29.045416 11779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:29.045742 11779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:29.046025 11779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:29.046382 11779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:29.046448 11779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.046490 11779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:29.046514 11779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.054349 11779 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:42755
I20260808 21:00:29.054402 11959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:42755 every 8 connection(s)
I20260808 21:00:29.054802 11779 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 21:00:29.059723 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 11779
I20260808 21:00:29.059883 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 21:00:29.060115 16290 external_mini_cluster.cc:1428] Setting key 7e53833b0cd3e9399160046b63209a4d
Aug 08 21:00:29 dist-test-slave-c12x krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222829, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.130@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:29.066179 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.055797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:40495 (local address 127.15.232.190:41747)
0808 21:00:29.055871 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.055873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.056146 (+ 273us) server_negotiation.cc:407] Connection header received
0808 21:00:29.056869 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.056871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.056885 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.056979 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:29.058092 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.058633 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.059762 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.060066 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.060150 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.062976 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:29.062977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:29.062978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:29.062994 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:29.064629 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:29.065085 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:29.065087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:29.065087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:29.065151 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:29.065434 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:29.065435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:29.065435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:29.065569 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:29.065650 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.065963 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.066071 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 21:00:29.066365 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.055780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41747 (local address 127.15.232.130:40495)
0808 21:00:29.056007 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.056047 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:29.056643 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:29.057072 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:29.057077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:29.057756 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:29.057974 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.057980 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.058747 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.058748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.059656 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.059659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.060158 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.060160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.060299 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.060934 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:29.060944 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:29.062850 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:29.064728 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:29.064737 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:29.064740 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:29.064999 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:29.065225 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:29.065225 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:29.065226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:29.065354 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:29.065639 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:29.065647 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:29.065810 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 21:00:29.065922 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 21:00:29.066104 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":122,"threads_started":1}
I20260808 21:00:29.066918 11960 heartbeater.cc:347] Connected to a master server at 127.15.232.190:41747
I20260808 21:00:29.067029 11960 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:29.067261 11960 heartbeater.cc:511] Master 127.15.232.190:41747 requested a full tablet report, sending...
I20260808 21:00:29.067767 11488 ts_manager.cc:194] Registered new tserver with Master: 64e2f38cbe444d3d975568fa868e52c8 (127.15.232.130:42755)
I20260808 21:00:29.068351 11488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.130@KRBTEST.COM'} at 127.15.232.130:40495
WARNING: no policy specified for kudu/127.15.232.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
Entry for principal kudu/127.15.232.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab.
I20260808 21:00:29.096786 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41747
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:29.191015 11970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:29.191248 11970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:29.191277 11970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:29.192557 11970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:29.192612 11970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:29.192642 11970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 21:00:29.194263 11970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.131.keytab
--principal=kudu/127.15.232.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:41747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:29.194502 11970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:29.194706 11970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:29.200954 11984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.200899 11981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.200902 11979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:29.201511 11970 server_base.cc:1034] running on GCE node
I20260808 21:00:29.201821 11970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:29.202625 11970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:29.204105 11970 hybrid_clock.cc:648] HybridClock initialized: now 1786222829204057 us; error 46 us; skew 500 ppm
Aug 08 21:00:29 dist-test-slave-c12x krb5kdc[11429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222829, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:29.207710 11970 init.cc:377] Logged in from keytab as kudu/127.15.232.131@KRBTEST.COM (short username kudu)
I20260808 21:00:29.208501 11970 webserver.cc:460] Webserver started at http://127.15.232.131:37933/ using document root <none> and password file <none>
I20260808 21:00:29.208875 11970 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:29.208962 11970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:29.209143 11970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:29.210420 11970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "adb8bb360adbc661854fd97cb0316537"
server_key_iv: "678309a0b4f77164fc8b5b38b166e9ab"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.210999 11970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "adb8bb360adbc661854fd97cb0316537"
server_key_iv: "678309a0b4f77164fc8b5b38b166e9ab"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.213032 11970 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 21:00:29.214565 11995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.215065 11970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:29.215209 11970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "adb8bb360adbc661854fd97cb0316537"
server_key_iv: "678309a0b4f77164fc8b5b38b166e9ab"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.215298 11970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:29.242944 11970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:29.245535 11970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:29.245672 11970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:29.245888 11970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:29.246269 11970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:29.246307 11970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.246331 11970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:29.246344 11970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.254384 11970 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:38621
I20260808 21:00:29.254462 12156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:38621 every 8 connection(s)
I20260808 21:00:29.254848 11970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
Aug 08 21:00:29 dist-test-slave-c12x krb5kdc[11429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222829, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.131@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:29.264534 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 11970
I20260808 21:00:29.264778 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 21:00:29.265059 16290 external_mini_cluster.cc:1428] Setting key 8792911c20f1ec4baf65f3569a1b4f1d
I20260808 21:00:29.269454 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.256127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:46961 (local address 127.15.232.190:41747)
0808 21:00:29.256228 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.256230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.256532 (+ 302us) server_negotiation.cc:407] Connection header received
0808 21:00:29.257063 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.257064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.257077 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.257148 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:29.258546 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.259225 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.260488 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.260688 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.260763 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.264380 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:29.264382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:29.264383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:29.264394 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:29.266764 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:29.267854 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:29.267856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:29.267856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:29.267926 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:29.268397 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:29.268399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:29.268400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:29.268627 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:29.268768 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.269238 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.269341 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 21:00:29.269565 12160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.256074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:41747 (local address 127.15.232.131:46961)
0808 21:00:29.256342 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.256382 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:29.256920 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:29.257284 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:29.257288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:29.258169 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:29.258397 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.258403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.259356 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.259357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.260367 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.260371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.260929 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.260931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.261146 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.261962 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:00:29.261973 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:29.264092 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:29.266934 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:29.266945 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:29.266949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:29.267731 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:29.268036 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:00:29.268037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:00:29.268037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:00:29.268195 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:00:29.268789 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:29.268797 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:29.268988 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 21:00:29.269124 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 21:00:29.269303 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":145,"threads_started":1}
I20260808 21:00:29.270373 12158 heartbeater.cc:347] Connected to a master server at 127.15.232.190:41747
I20260808 21:00:29.270550 12158 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:29.270939 12158 heartbeater.cc:511] Master 127.15.232.190:41747 requested a full tablet report, sending...
I20260808 21:00:29.271720 11488 ts_manager.cc:194] Registered new tserver with Master: 88a5990a3e2444bb9f8e4cedf3dc1bd7 (127.15.232.131:38621)
I20260808 21:00:29.272224 11488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.232.131@KRBTEST.COM'} at 127.15.232.131:46961
I20260808 21:00:29.277253 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 21:00:29.287103 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.278803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51928 (local address 127.15.232.190:41747)
0808 21:00:29.278879 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.278881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.278953 (+ 72us) server_negotiation.cc:407] Connection header received
0808 21:00:29.278977 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.278977 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.278987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.279034 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:29.279967 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.280515 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.281372 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.281606 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.281669 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.283191 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:29.283192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:29.283192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:29.283200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:29.285283 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:29.285887 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:29.285889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:29.285890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:29.286020 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:29.286334 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:29.286334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:29.286335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:29.286574 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:29.286706 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.286916 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.287021 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 21:00:29.290123 11488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 21:00:29.292178 11488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 21:00:29.299713 12171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.294985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:38621 (local address 127.0.0.1:58844)
0808 21:00:29.295361 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.295406 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:29.295503 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:29.295882 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:29.295883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:29.295886 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:29.295952 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.295955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.296910 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.296911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.298382 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.298386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.299270 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.299272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.299370 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.299389 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 21:00:29.299523 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 21:00:29.299597 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":73,"threads_started":1}
I20260808 21:00:29.299798 12173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.295113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.15.232.130:42755)
0808 21:00:29.295329 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.295335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.295495 (+ 160us) server_negotiation.cc:407] Connection header received
0808 21:00:29.295619 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.295621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.295639 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.295698 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:29.296000 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.297151 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.298478 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.299209 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.299288 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.299342 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.299658 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.299707 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":95,"threads_started":1}
I20260808 21:00:29.299798 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.295401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35240 (local address 127.15.232.129:46507)
0808 21:00:29.295828 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.295832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.295842 (+ 10us) server_negotiation.cc:407] Connection header received
0808 21:00:29.295880 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.295882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.295897 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.296114 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:29.296207 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.297254 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.298674 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.299403 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.299520 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.299579 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.299658 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.299702 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":139,"threads_started":1}
I20260808 21:00:29.299830 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.295225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:46507 (local address 127.0.0.1:35240)
0808 21:00:29.295634 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.295643 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:29.295683 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:29.295998 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:29.295998 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:29.296001 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:29.296039 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.296041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.297376 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.297377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.298572 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.298575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.299520 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.299522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.299583 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.299587 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:29.299647 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 21:00:29.299687 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":73,"threads_started":1}
I20260808 21:00:29.299996 12172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.295002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:42755 (local address 127.0.0.1:58894)
0808 21:00:29.295377 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.295406 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:29.295521 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:29.295821 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:29.295824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:29.295827 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:29.295907 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.295911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.297279 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.297280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.298375 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.298378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.299382 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.299383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.299493 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.299499 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 21:00:29.299547 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 21:00:29.299599 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":40,"thread_start_us":105,"threads_started":1}
I20260808 21:00:29.300283 12174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.295111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.15.232.131:38621)
0808 21:00:29.295603 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.295608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.295620 (+ 12us) server_negotiation.cc:407] Connection header received
0808 21:00:29.295663 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.295669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.295683 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.295785 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:29.296050 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.296791 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.298517 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.299159 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.299272 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.299342 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.300092 (+ 750us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.300167 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":173,"threads_started":1}
I20260808 21:00:29.300693 11677 tablet_service.cc:1467] Processing CreateTablet for tablet e34957f98323479299e9fdc6e11667c8 (DEFAULT_TABLE table=test-table [id=706b0025c7a340d899443e8d4def3da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:29.301126 11677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e34957f98323479299e9fdc6e11667c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:29.301088 11864 tablet_service.cc:1467] Processing CreateTablet for tablet e34957f98323479299e9fdc6e11667c8 (DEFAULT_TABLE table=test-table [id=706b0025c7a340d899443e8d4def3da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:29.301137 12060 tablet_service.cc:1467] Processing CreateTablet for tablet e34957f98323479299e9fdc6e11667c8 (DEFAULT_TABLE table=test-table [id=706b0025c7a340d899443e8d4def3da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:29.301424 11864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e34957f98323479299e9fdc6e11667c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:29.301564 12060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e34957f98323479299e9fdc6e11667c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:29.305063 12181 tablet_bootstrap.cc:492] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: Bootstrap starting.
I20260808 21:00:29.305042 12178 tablet_bootstrap.cc:492] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: Bootstrap starting.
I20260808 21:00:29.305720 12181 tablet_bootstrap.cc:654] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:29.305817 12178 tablet_bootstrap.cc:654] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:29.306269 12181 log.cc:824] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:29.306401 12178 log.cc:824] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:29.306684 12179 tablet_bootstrap.cc:492] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792: Bootstrap starting.
I20260808 21:00:29.307332 12178 tablet_bootstrap.cc:492] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: No bootstrap required, opened a new log
I20260808 21:00:29.307391 12179 tablet_bootstrap.cc:654] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:29.307508 12178 ts_tablet_manager.cc:1397] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 21:00:29.307843 12181 tablet_bootstrap.cc:492] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: No bootstrap required, opened a new log
I20260808 21:00:29.307902 12181 ts_tablet_manager.cc:1397] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 21:00:29.309171 12178 raft_consensus.cc:348] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } }
I20260808 21:00:29.309371 12178 raft_consensus.cc:374] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:29.309435 12178 raft_consensus.cc:729] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88a5990a3e2444bb9f8e4cedf3dc1bd7, State: Initialized, Role: FOLLOWER
I20260808 21:00:29.309311 12181 raft_consensus.cc:348] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } }
I20260808 21:00:29.309495 12181 raft_consensus.cc:374] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:29.309543 12181 raft_consensus.cc:729] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e2f38cbe444d3d975568fa868e52c8, State: Initialized, Role: FOLLOWER
I20260808 21:00:29.309614 12178 consensus_queue.cc:260] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } }
I20260808 21:00:29.309687 12181 consensus_queue.cc:260] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } }
I20260808 21:00:29.310004 12158 heartbeater.cc:503] Master 127.15.232.190:41747 was elected leader, sending a full tablet report...
I20260808 21:00:29.310002 12178 ts_tablet_manager.cc:1428] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:29.310040 11960 heartbeater.cc:503] Master 127.15.232.190:41747 was elected leader, sending a full tablet report...
I20260808 21:00:29.310724 12179 log.cc:824] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:29.310041 12181 ts_tablet_manager.cc:1428] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:29.311897 12179 tablet_bootstrap.cc:492] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792: No bootstrap required, opened a new log
I20260808 21:00:29.311982 12179 ts_tablet_manager.cc:1397] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 21:00:29.313817 12179 raft_consensus.cc:348] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } }
I20260808 21:00:29.314044 12179 raft_consensus.cc:374] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:29.314121 12179 raft_consensus.cc:729] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b8c5fdc55e4ecd92d2148c00c31792, State: Initialized, Role: FOLLOWER
I20260808 21:00:29.314257 12179 consensus_queue.cc:260] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } }
I20260808 21:00:29.314602 12179 ts_tablet_manager.cc:1428] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 21:00:29.314630 11769 heartbeater.cc:503] Master 127.15.232.190:41747 was elected leader, sending a full tablet report...
W20260808 21:00:29.348273 11770 tablet.cc:2369] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 21:00:29.384296 12186 raft_consensus.cc:482] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:29.384506 12186 raft_consensus.cc:504] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } }
I20260808 21:00:29.385037 12186 leader_election.cc:288] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88a5990a3e2444bb9f8e4cedf3dc1bd7 (127.15.232.131:38621), 89b8c5fdc55e4ecd92d2148c00c31792 (127.15.232.129:46507)
I20260808 21:00:29.388293 12174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.385463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:39063 (local address 127.15.232.131:38621)
0808 21:00:29.385542 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.385545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.385558 (+ 13us) server_negotiation.cc:407] Connection header received
0808 21:00:29.385596 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.385597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.385615 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.385722 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:29.385972 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.386455 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.387527 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.387911 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.388010 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.388019 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.388184 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.388223 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 21:00:29.388296 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.385327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:38621 (local address 127.15.232.130:39063)
0808 21:00:29.385399 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.385413 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:29.385464 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:29.385765 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:29.385766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:29.385768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:29.385854 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.385856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.386556 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.386557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.387361 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.387363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.388009 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.388010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.388096 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.388099 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 21:00:29.388165 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 21:00:29.388213 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":22}
I20260808 21:00:29.388659 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.385604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:34313 (local address 127.15.232.129:46507)
0808 21:00:29.385678 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.385681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.385759 (+ 78us) server_negotiation.cc:407] Connection header received
0808 21:00:29.385870 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.385872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.385885 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.385968 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:29.386159 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.386688 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.387838 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.388241 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.388338 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.388346 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.388531 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.388578 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 21:00:29.388831 12086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e34957f98323479299e9fdc6e11667c8" candidate_uuid: "64e2f38cbe444d3d975568fa868e52c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" is_pre_election: true
I20260808 21:00:29.388923 12191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.385499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:46507 (local address 127.15.232.130:34313)
0808 21:00:29.385693 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.385705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:29.385784 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:29.385980 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:29.385981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:29.385983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:29.386069 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.386071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.386791 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.386792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.387728 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.387731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.388340 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.388342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.388431 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.388438 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 21:00:29.388775 (+ 337us) client_negotiation.cc:240] Negotiation successful
0808 21:00:29.388826 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":24,"thread_start_us":129,"threads_started":1}
I20260808 21:00:29.389037 12086 raft_consensus.cc:2454] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64e2f38cbe444d3d975568fa868e52c8 in term 0.
I20260808 21:00:29.389289 11703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e34957f98323479299e9fdc6e11667c8" candidate_uuid: "64e2f38cbe444d3d975568fa868e52c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" is_pre_election: true
I20260808 21:00:29.389355 11802 leader_election.cc:302] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64e2f38cbe444d3d975568fa868e52c8, 88a5990a3e2444bb9f8e4cedf3dc1bd7; no voters:
I20260808 21:00:29.389480 11703 raft_consensus.cc:2454] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64e2f38cbe444d3d975568fa868e52c8 in term 0.
I20260808 21:00:29.389546 12186 raft_consensus.cc:2781] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 21:00:29.389623 12186 raft_consensus.cc:482] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 21:00:29.389657 12186 raft_consensus.cc:3037] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:29.390534 12186 raft_consensus.cc:504] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } }
I20260808 21:00:29.390658 12186 leader_election.cc:288] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [CANDIDATE]: Term 1 election: Requested vote from peers 88a5990a3e2444bb9f8e4cedf3dc1bd7 (127.15.232.131:38621), 89b8c5fdc55e4ecd92d2148c00c31792 (127.15.232.129:46507)
I20260808 21:00:29.390983 12086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e34957f98323479299e9fdc6e11667c8" candidate_uuid: "64e2f38cbe444d3d975568fa868e52c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7"
I20260808 21:00:29.391004 11703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e34957f98323479299e9fdc6e11667c8" candidate_uuid: "64e2f38cbe444d3d975568fa868e52c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b8c5fdc55e4ecd92d2148c00c31792"
I20260808 21:00:29.391077 12086 raft_consensus.cc:3037] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:29.391119 11703 raft_consensus.cc:3037] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:29.391968 12086 raft_consensus.cc:2454] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64e2f38cbe444d3d975568fa868e52c8 in term 1.
I20260808 21:00:29.392091 11703 raft_consensus.cc:2454] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64e2f38cbe444d3d975568fa868e52c8 in term 1.
I20260808 21:00:29.392144 11802 leader_election.cc:302] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64e2f38cbe444d3d975568fa868e52c8, 88a5990a3e2444bb9f8e4cedf3dc1bd7; no voters:
I20260808 21:00:29.392256 12186 raft_consensus.cc:2781] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:29.392405 12186 raft_consensus.cc:686] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 1 LEADER]: Becoming Leader. State: Replica: 64e2f38cbe444d3d975568fa868e52c8, State: Running, Role: LEADER
I20260808 21:00:29.392527 12186 consensus_queue.cc:237] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } }
I20260808 21:00:29.393553 11486 catalog_manager.cc:5495] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64e2f38cbe444d3d975568fa868e52c8 (127.15.232.130). New cstate: current_term: 1 leader_uuid: "64e2f38cbe444d3d975568fa868e52c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e2f38cbe444d3d975568fa868e52c8" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 42755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 } health_report { overall_health: UNKNOWN } } }
I20260808 21:00:29.418825 12173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.415876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.15.232.130:42755)
0808 21:00:29.415977 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.415980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.415990 (+ 10us) server_negotiation.cc:407] Connection header received
0808 21:00:29.416031 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.416031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.416051 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.416127 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:00:29.416309 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.416824 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.417809 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.418025 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.418090 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.418454 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:00:29.418579 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:00:29.418672 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.418732 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.418755 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 21:00:29.421052 11703 raft_consensus.cc:1261] T e34957f98323479299e9fdc6e11667c8 P 89b8c5fdc55e4ecd92d2148c00c31792 [term 1 FOLLOWER]: Refusing update from remote peer 64e2f38cbe444d3d975568fa868e52c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:29.421052 12086 raft_consensus.cc:1261] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [term 1 FOLLOWER]: Refusing update from remote peer 64e2f38cbe444d3d975568fa868e52c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:29.421424 12186 consensus_queue.cc:1035] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88a5990a3e2444bb9f8e4cedf3dc1bd7" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 38621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:29.421590 12193 consensus_queue.cc:1035] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89b8c5fdc55e4ecd92d2148c00c31792" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:29.424398 12198 mvcc.cc:205] Tried to move back new op lower bound from 7316368709305511936 to 7316368709192613888. Current Snapshot: MvccSnapshot[applied={T|T < 7316368709305511936}]
I20260808 21:00:29.426564 11486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:29.426679 11486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:29.427592 11486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caf88762d128447c983762220aaf549f: Sending DeleteTablet for 3 replicas of tablet e34957f98323479299e9fdc6e11667c8
I20260808 21:00:29.427970 11864 tablet_service.cc:1514] Processing DeleteTablet for tablet e34957f98323479299e9fdc6e11667c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:29 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:58894
I20260808 21:00:29.428010 11677 tablet_service.cc:1514] Processing DeleteTablet for tablet e34957f98323479299e9fdc6e11667c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:29 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:35240
I20260808 21:00:29.428298 12207 tablet_replica.cc:330] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: stopping tablet replica
I20260808 21:00:29.428313 12060 tablet_service.cc:1514] Processing DeleteTablet for tablet e34957f98323479299e9fdc6e11667c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:29 UTC) from {username='kudu', principal='kudu/127.15.232.190@KRBTEST.COM'} at 127.0.0.1:58844
I20260808 21:00:29.428419 12207 raft_consensus.cc:2229] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 21:00:29.428568 12210 tablet_replica.cc:330] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: stopping tablet replica
I20260808 21:00:29.428612 12207 raft_consensus.cc:2258] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:29.428714 12210 raft_consensus.cc:2229] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:29.428856 12210 raft_consensus.cc:2258] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:29.428917 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 11586
I20260808 21:00:29.429756 12207 ts_tablet_manager.cc:1905] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:29.432149 12207 ts_tablet_manager.cc:1918] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:29.432240 12207 log.cc:1196] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/e34957f98323479299e9fdc6e11667c8
I20260808 21:00:29.432575 12207 ts_tablet_manager.cc:1939] T e34957f98323479299e9fdc6e11667c8 P 64e2f38cbe444d3d975568fa868e52c8: Deleting consensus metadata
I20260808 21:00:29.433143 11466 catalog_manager.cc:4841] TS 64e2f38cbe444d3d975568fa868e52c8 (127.15.232.130:42755): tablet e34957f98323479299e9fdc6e11667c8 (table test-table [id=706b0025c7a340d899443e8d4def3da5]) successfully deleted
I20260808 21:00:29.448698 12210 ts_tablet_manager.cc:1905] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:29.450860 12210 ts_tablet_manager.cc:1918] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:29.450991 12210 log.cc:1196] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/e34957f98323479299e9fdc6e11667c8
I20260808 21:00:29.451381 12210 ts_tablet_manager.cc:1939] T e34957f98323479299e9fdc6e11667c8 P 88a5990a3e2444bb9f8e4cedf3dc1bd7: Deleting consensus metadata
I20260808 21:00:29.452065 11465 catalog_manager.cc:4841] TS 88a5990a3e2444bb9f8e4cedf3dc1bd7 (127.15.232.131:38621): tablet e34957f98323479299e9fdc6e11667c8 (table test-table [id=706b0025c7a340d899443e8d4def3da5]) successfully deleted
W20260808 21:00:29.453289 11465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:46507 (error 108)
I20260808 21:00:29.454000 12172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.453827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:46507 (local address 127.0.0.1:35254)
0808 21:00:29.453905 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.453955 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:46507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 21:00:29.454105 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 11779
W20260808 21:00:29.460289 11465 catalog_manager.cc:4584] TS 89b8c5fdc55e4ecd92d2148c00c31792 (127.15.232.129:46507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e34957f98323479299e9fdc6e11667c8: Network error: Client connection negotiation failed: client connection to 127.15.232.129:46507: connect: Connection refused (error 111)
I20260808 21:00:29.467036 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 11970
I20260808 21:00:29.478304 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 11444
2026-08-08T21:00:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2488 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T21:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:29Z Disabled control of system clock
I20260808 21:00:29.519873 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:40033
--webserver_interface=127.15.232.190
--webserver_port=0
--builtin_ntp_servers=127.15.232.148:44853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:40033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 21:00:29.606674 12218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:29.606915 12218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:29.606940 12218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:29.608083 12218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:29.608125 12218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:29.608139 12218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:29.608151 12218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:29.608163 12218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:29.609485 12218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:40033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:40033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:29.609707 12218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:29.609956 12218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:29.615276 12225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.615324 12227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.615648 12224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:29.615657 12218 server_base.cc:1034] running on GCE node
I20260808 21:00:29.616145 12218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:29.616602 12218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:29.617851 12218 hybrid_clock.cc:648] HybridClock initialized: now 1786222829617825 us; error 50 us; skew 500 ppm
I20260808 21:00:29.619872 12218 webserver.cc:460] Webserver started at http://127.15.232.190:39735/ using document root <none> and password file <none>
I20260808 21:00:29.620255 12218 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:29.620374 12218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:29.620553 12218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:29.622128 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "284d8e9907fc480ab17256c6ed914f33"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "f0531bc5bd586f7b0c6c50dbb1fb5294"
server_key_iv: "3cf8c37d3779c05859e7a4b6970118ad"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.622627 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "284d8e9907fc480ab17256c6ed914f33"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "f0531bc5bd586f7b0c6c50dbb1fb5294"
server_key_iv: "3cf8c37d3779c05859e7a4b6970118ad"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.624859 12218 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260808 21:00:29.625880 12235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.626205 12218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:29.626333 12218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "284d8e9907fc480ab17256c6ed914f33"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "f0531bc5bd586f7b0c6c50dbb1fb5294"
server_key_iv: "3cf8c37d3779c05859e7a4b6970118ad"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.626487 12218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:29.642459 12218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:29.642858 12218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:29.643040 12218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:29.648169 12218 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:40033
I20260808 21:00:29.648339 12307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:40033 every 8 connection(s)
I20260808 21:00:29.648650 12218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:29.649825 12308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:29.652475 12308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33: Bootstrap starting.
I20260808 21:00:29.653088 12308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:29.653360 12308 log.cc:824] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:29.654188 12308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33: No bootstrap required, opened a new log
I20260808 21:00:29.656036 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 12218
I20260808 21:00:29.656196 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:29.656106 12308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d8e9907fc480ab17256c6ed914f33" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40033 } }
I20260808 21:00:29.656333 12308 raft_consensus.cc:374] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:29.656387 12308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284d8e9907fc480ab17256c6ed914f33, State: Initialized, Role: FOLLOWER
I20260808 21:00:29.656524 16290 external_mini_cluster.cc:1428] Setting key da7931ef9772455126467af19bd178be
I20260808 21:00:29.656580 12308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d8e9907fc480ab17256c6ed914f33" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40033 } }
I20260808 21:00:29.656699 12308 raft_consensus.cc:388] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:29.656751 12308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:29.656806 12308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:29.657642 12308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d8e9907fc480ab17256c6ed914f33" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40033 } }
I20260808 21:00:29.657779 12308 leader_election.cc:302] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 284d8e9907fc480ab17256c6ed914f33; no voters:
I20260808 21:00:29.658072 12308 leader_election.cc:288] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:29.658106 12317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:29.658320 12317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [term 1 LEADER]: Becoming Leader. State: Replica: 284d8e9907fc480ab17256c6ed914f33, State: Running, Role: LEADER
I20260808 21:00:29.658469 12308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:29.658423 12317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d8e9907fc480ab17256c6ed914f33" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40033 } }
I20260808 21:00:29.658922 12317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 284d8e9907fc480ab17256c6ed914f33. Latest consensus state: current_term: 1 leader_uuid: "284d8e9907fc480ab17256c6ed914f33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d8e9907fc480ab17256c6ed914f33" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40033 } } }
I20260808 21:00:29.659128 12317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:29.659677 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "284d8e9907fc480ab17256c6ed914f33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284d8e9907fc480ab17256c6ed914f33" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 40033 } } }
I20260808 21:00:29.659770 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33 [sys.catalog]: This master's current role is: LEADER
W20260808 21:00:29.660527 12335 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:00:29.660588 12335 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 21:00:29.660635 12327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:29.660791 12327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:29.662571 12327 catalog_manager.cc:1347] Generated new cluster ID: 781ab9410569482cadd62969424de976
I20260808 21:00:29.662629 12327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:29.664019 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.657162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47418 (local address 127.15.232.190:40033)
0808 21:00:29.657586 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.657604 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.657633 (+ 29us) server_negotiation.cc:407] Connection header received
0808 21:00:29.657749 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.657755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.657888 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.658081 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:29.658584 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.660580 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.661587 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.662246 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.662407 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.663183 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:29.663189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:29.663190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:29.663379 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:29.663415 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:29.663426 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:29.663605 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.663698 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.663793 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":299,"threads_started":1}
I20260808 21:00:29.672281 12327 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:29.672905 12327 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:29.684098 12327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33: Generated new TSK 0
I20260808 21:00:29.684386 12327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:29.693858 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.129:0
--local_ip_for_outbound_sockets=127.15.232.129
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40033
--builtin_ntp_servers=127.15.232.148:44853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 21:00:29.786550 12340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:29.786778 12340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:29.786803 12340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:29.788103 12340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:29.788151 12340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:29.788177 12340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.129
I20260808 21:00:29.789628 12340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.232.129
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.129
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:29.789924 12340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:29.790269 12340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:29.797271 12349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.797288 12350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.797282 12352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:29.797549 12340 server_base.cc:1034] running on GCE node
I20260808 21:00:29.797793 12340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:29.798090 12340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:29.799300 12340 hybrid_clock.cc:648] HybridClock initialized: now 1786222829799276 us; error 40 us; skew 500 ppm
I20260808 21:00:29.801210 12340 webserver.cc:460] Webserver started at http://127.15.232.129:36285/ using document root <none> and password file <none>
I20260808 21:00:29.801496 12340 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:29.801575 12340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:29.801733 12340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:29.802892 12340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/data/instance:
uuid: "ccd4d478356646baa2dd8bc74c106d0b"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "ae7b394d02b5b96692ac05aa188ab540"
server_key_iv: "aee2d8da96a5a9efebb25283329e575d"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.803297 12340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance:
uuid: "ccd4d478356646baa2dd8bc74c106d0b"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "ae7b394d02b5b96692ac05aa188ab540"
server_key_iv: "aee2d8da96a5a9efebb25283329e575d"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.804801 12340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:29.805670 12362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.805914 12340 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:29.806057 12340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/wal
uuid: "ccd4d478356646baa2dd8bc74c106d0b"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "ae7b394d02b5b96692ac05aa188ab540"
server_key_iv: "aee2d8da96a5a9efebb25283329e575d"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.806156 12340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:29.817341 12340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:29.817741 12340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:29.817917 12340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:29.818181 12340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:29.818562 12340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:29.818619 12340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.818666 12340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:29.818696 12340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.825192 12340 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.129:36315
I20260808 21:00:29.825237 12517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.129:36315 every 8 connection(s)
I20260808 21:00:29.825604 12340 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/data/info.pb
I20260808 21:00:29.831786 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 12340
I20260808 21:00:29.831880 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.826486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.129:46277 (local address 127.15.232.190:40033)
0808 21:00:29.826546 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.826548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.826808 (+ 260us) server_negotiation.cc:407] Connection header received
0808 21:00:29.827320 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.827321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.827335 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.827399 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:29.828239 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.828770 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.829930 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.830152 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.830213 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.831479 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:29.831484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:29.831484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:29.831492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:29.831509 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:29.831513 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:29.831565 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.831772 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.831809 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 21:00:29.832002 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-0/wal/instance
I20260808 21:00:29.832262 16290 external_mini_cluster.cc:1428] Setting key 84511367289f934cb8862f8032a09f6a
I20260808 21:00:29.832311 12520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.826447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:40033 (local address 127.15.232.129:46277)
0808 21:00:29.826672 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.826707 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:29.827209 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:29.827555 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:29.827558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:29.827567 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:29.828132 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.828137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.828859 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.828860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.829831 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.829834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.830276 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.830277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.830408 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.831126 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 21:00:29.831128 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:29.831181 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 21:00:29.831236 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:29.831602 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:29.831611 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:29.831616 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 21:00:29.831875 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 21:00:29.832056 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":118,"threads_started":1}
I20260808 21:00:29.832942 12518 heartbeater.cc:347] Connected to a master server at 127.15.232.190:40033
I20260808 21:00:29.833087 12518 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:29.833309 12518 heartbeater.cc:511] Master 127.15.232.190:40033 requested a full tablet report, sending...
I20260808 21:00:29.833670 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.130:0
--local_ip_for_outbound_sockets=127.15.232.130
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40033
--builtin_ntp_servers=127.15.232.148:44853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 21:00:29.833976 12258 ts_manager.cc:194] Registered new tserver with Master: ccd4d478356646baa2dd8bc74c106d0b (127.15.232.129:36315)
I20260808 21:00:29.834638 12258 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.129:46277
W20260808 21:00:29.925508 12521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:29.925740 12521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:29.925763 12521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:29.927022 12521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:29.927073 12521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:29.927101 12521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.130
I20260808 21:00:29.928612 12521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.232.130
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.130
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:29.928893 12521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:29.929325 12521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 21:00:29.935297 12521 server_base.cc:1034] running on GCE node
W20260808 21:00:29.935245 12536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.935210 12529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:29.935235 12528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:29.935743 12521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:29.935994 12521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:29.937160 12521 hybrid_clock.cc:648] HybridClock initialized: now 1786222829937145 us; error 34 us; skew 500 ppm
I20260808 21:00:29.938702 12521 webserver.cc:460] Webserver started at http://127.15.232.130:38831/ using document root <none> and password file <none>
I20260808 21:00:29.939015 12521 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:29.939075 12521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:29.939260 12521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:29.940500 12521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/data/instance:
uuid: "da7a75e7c73b4e32ac5582846e1ed5b5"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "002855eeda52851e825314e817f13cb2"
server_key_iv: "dcbb3796f8cf102d8fe9cd2a678a7371"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.940903 12521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance:
uuid: "da7a75e7c73b4e32ac5582846e1ed5b5"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "002855eeda52851e825314e817f13cb2"
server_key_iv: "dcbb3796f8cf102d8fe9cd2a678a7371"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.942543 12521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 21:00:29.943698 12544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.944156 12521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 21:00:29.944370 12521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/wal
uuid: "da7a75e7c73b4e32ac5582846e1ed5b5"
format_stamp: "Formatted at 2026-08-08 21:00:29 on dist-test-slave-c12x"
server_key: "002855eeda52851e825314e817f13cb2"
server_key_iv: "dcbb3796f8cf102d8fe9cd2a678a7371"
server_key_version: "encryptionkey@0"
I20260808 21:00:29.944465 12521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:29.957331 12521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:29.957769 12521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:29.957988 12521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:29.958261 12521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:29.958755 12521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:29.958806 12521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.958837 12521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:29.958891 12521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:29.966773 12521 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.130:46531
I20260808 21:00:29.966822 12703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.130:46531 every 8 connection(s)
I20260808 21:00:29.967250 12521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/data/info.pb
I20260808 21:00:29.970747 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 12521
I20260808 21:00:29.970877 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/wal/instance
I20260808 21:00:29.971097 16290 external_mini_cluster.cc:1428] Setting key 2a027fc4f078af34a8793ec23ddb1698
I20260808 21:00:29.972364 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.232.131:0
--local_ip_for_outbound_sockets=127.15.232.131
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40033
--builtin_ntp_servers=127.15.232.148:44853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 21:00:29.974593 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.968415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:54947 (local address 127.15.232.190:40033)
0808 21:00:29.968465 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:29.968467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:29.968742 (+ 275us) server_negotiation.cc:407] Connection header received
0808 21:00:29.969354 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:29.969355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:29.969368 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:29.969420 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:29.970150 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.970765 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.972369 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.972677 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.972764 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.974104 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:29.974106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:29.974106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:29.974113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:29.974134 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:29.974138 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:29.974202 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:29.974466 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 21:00:29.974528 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 21:00:29.974785 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:29.968419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:40033 (local address 127.15.232.130:54947)
0808 21:00:29.968616 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:29.968652 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:29.969226 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:29.969505 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:29.969508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:29.969517 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:29.970040 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.970045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.970930 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.970931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.972265 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:29.972269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:29.972831 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:29.972832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:29.972970 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:29.973625 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 21:00:29.973626 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:29.973658 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 21:00:29.973939 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:29.974232 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:29.974239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:29.974243 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:29.974337 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 21:00:29.974480 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260808 21:00:29.975430 12705 heartbeater.cc:347] Connected to a master server at 127.15.232.190:40033
I20260808 21:00:29.975595 12705 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:29.975888 12705 heartbeater.cc:511] Master 127.15.232.190:40033 requested a full tablet report, sending...
I20260808 21:00:29.976421 12258 ts_manager.cc:194] Registered new tserver with Master: da7a75e7c73b4e32ac5582846e1ed5b5 (127.15.232.130:46531)
I20260808 21:00:29.976878 12258 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.130:54947
W20260808 21:00:30.060513 12708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:30.060775 12708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:30.060815 12708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:30.062059 12708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:30.062139 12708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:30.062186 12708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.232.131
I20260808 21:00:30.063752 12708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.232.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.232.131
--webserver_port=0
--tserver_master_addrs=127.15.232.190:40033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.232.131
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:30.064054 12708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:30.064386 12708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:30.070253 12714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:30.070366 12708 server_base.cc:1034] running on GCE node
W20260808 21:00:30.070329 12721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:30.070592 12715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:30.070849 12708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:30.071146 12708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:30.072366 12708 hybrid_clock.cc:648] HybridClock initialized: now 1786222830072346 us; error 28 us; skew 500 ppm
I20260808 21:00:30.073814 12708 webserver.cc:460] Webserver started at http://127.15.232.131:44021/ using document root <none> and password file <none>
I20260808 21:00:30.074095 12708 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:30.074172 12708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:30.074333 12708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:30.075536 12708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/data/instance:
uuid: "a6ed1019f50f409b8a63d46fe5b03afb"
format_stamp: "Formatted at 2026-08-08 21:00:30 on dist-test-slave-c12x"
server_key: "b2eba3cac7db20b27821a12a838b4555"
server_key_iv: "5c32453f78ac4471a76c0198c299ba8d"
server_key_version: "encryptionkey@0"
I20260808 21:00:30.075933 12708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance:
uuid: "a6ed1019f50f409b8a63d46fe5b03afb"
format_stamp: "Formatted at 2026-08-08 21:00:30 on dist-test-slave-c12x"
server_key: "b2eba3cac7db20b27821a12a838b4555"
server_key_iv: "5c32453f78ac4471a76c0198c299ba8d"
server_key_version: "encryptionkey@0"
I20260808 21:00:30.077396 12708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 21:00:30.078289 12728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:30.078549 12708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:30.078683 12708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/wal
uuid: "a6ed1019f50f409b8a63d46fe5b03afb"
format_stamp: "Formatted at 2026-08-08 21:00:30 on dist-test-slave-c12x"
server_key: "b2eba3cac7db20b27821a12a838b4555"
server_key_iv: "5c32453f78ac4471a76c0198c299ba8d"
server_key_version: "encryptionkey@0"
I20260808 21:00:30.078866 12708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:30.094087 12708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:30.094480 12708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:30.094693 12708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:30.094990 12708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 21:00:30.095366 12708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 21:00:30.095423 12708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:30.095507 12708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 21:00:30.095558 12708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:30.101856 12708 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.131:40369
I20260808 21:00:30.101939 12903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.131:40369 every 8 connection(s)
I20260808 21:00:30.102309 12708 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/data/info.pb
I20260808 21:00:30.108533 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.103206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.131:38039 (local address 127.15.232.190:40033)
0808 21:00:30.103269 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:30.103271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:30.103543 (+ 272us) server_negotiation.cc:407] Connection header received
0808 21:00:30.104327 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:30.104328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:30.104340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:30.104442 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:30.105172 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.105749 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.106889 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.107125 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.107205 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.108226 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:30.108227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:30.108227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:30.108232 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:30.108248 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:30.108252 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:30.108308 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:30.108403 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 21:00:30.108431 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 21:00:30.108952 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.103148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:40033 (local address 127.15.232.131:38039)
0808 21:00:30.103369 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:30.103417 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:30.104198 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:30.104519 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:30.104522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:30.104532 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:00:30.105058 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.105062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.105884 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.105885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.106759 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.106762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.107229 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.107230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.107350 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.108072 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 21:00:30.108073 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:00:30.108104 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 21:00:30.108130 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:00:30.108319 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:00:30.108324 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:00:30.108327 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 21:00:30.108384 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 21:00:30.108477 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":109,"threads_started":1}
I20260808 21:00:30.109730 12904 heartbeater.cc:347] Connected to a master server at 127.15.232.190:40033
I20260808 21:00:30.109862 12904 heartbeater.cc:464] Registering TS with master...
I20260808 21:00:30.110085 12904 heartbeater.cc:511] Master 127.15.232.190:40033 requested a full tablet report, sending...
I20260808 21:00:30.110116 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 12708
I20260808 21:00:30.110235 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/wal/instance
I20260808 21:00:30.110541 16290 external_mini_cluster.cc:1428] Setting key 98c189e0edf10a98520b8b00a9a16f7f
I20260808 21:00:30.110635 12258 ts_manager.cc:194] Registered new tserver with Master: a6ed1019f50f409b8a63d46fe5b03afb (127.15.232.131:40369)
I20260808 21:00:30.111135 12258 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.232.131:38039
I20260808 21:00:30.111727 16290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 21:00:30.116856 12314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.113366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47434 (local address 127.15.232.190:40033)
0808 21:00:30.113421 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:30.113423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:30.113480 (+ 57us) server_negotiation.cc:407] Connection header received
0808 21:00:30.113571 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:30.113571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:30.113585 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:30.113629 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:30.113821 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.114320 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.115262 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.115527 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.115603 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.116539 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:30.116540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:30.116541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:30.116546 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:30.116563 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:30.116568 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:30.116619 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:30.116729 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 21:00:30.116761 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 21:00:30.119199 12258 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 21:00:30.121109 12258 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 21:00:30.128237 12915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.123792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:40369 (local address 127.0.0.1:33584)
0808 21:00:30.124015 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:30.124056 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:30.124128 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:30.124448 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:30.124451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:30.124454 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:30.124537 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.124540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.126003 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.126004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.127116 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.127118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.127913 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.127914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.128006 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.128020 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 21:00:30.128096 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 21:00:30.128155 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":52,"thread_start_us":105,"threads_started":1}
I20260808 21:00:30.128248 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.124023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33584 (local address 127.15.232.131:40369)
0808 21:00:30.124226 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:30.124230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:30.124239 (+ 9us) server_negotiation.cc:407] Connection header received
0808 21:00:30.124287 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:30.124288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:30.124300 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:30.124382 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:30.124624 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.125911 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.127213 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.127809 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.127913 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.128011 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:30.128111 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 21:00:30.128151 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
I20260808 21:00:30.128573 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.123818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:36315 (local address 127.0.0.1:49524)
0808 21:00:30.124483 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:30.124493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:30.124548 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:30.124886 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:30.124887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:30.124889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:30.124929 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.124931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.126139 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.126141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.127132 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.127135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.128342 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.128343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.128412 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.128415 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 21:00:30.128472 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 21:00:30.128516 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":20,"thread_start_us":115,"threads_started":1}
I20260808 21:00:30.128436 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.123792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.130:46531 (local address 127.0.0.1:34388)
0808 21:00:30.124333 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:30.124342 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:30.124420 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:30.124671 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:30.124672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:30.124674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:30.124726 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.124728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.125678 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.125680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.126973 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.126976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.128180 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.128181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.128246 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.128250 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 21:00:30.128298 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 21:00:30.128345 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":4,"thread_start_us":115,"threads_started":1}
I20260808 21:00:30.128676 12918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.123999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.15.232.130:46531)
0808 21:00:30.124206 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:30.124211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:30.124414 (+ 203us) server_negotiation.cc:407] Connection header received
0808 21:00:30.124504 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:30.124506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:30.124521 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:30.124595 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:30.124809 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.125587 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.127078 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.127965 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.128047 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.128137 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:30.128525 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 21:00:30.128573 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260808 21:00:30.128950 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.124089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.15.232.129:36315)
0808 21:00:30.124275 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:30.124279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:30.124681 (+ 402us) server_negotiation.cc:407] Connection header received
0808 21:00:30.124730 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:30.124733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:30.124749 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:30.124821 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:30.125016 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.126055 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.127332 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.128158 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.128227 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.128310 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:30.128784 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 21:00:30.128863 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260808 21:00:30.129506 12614 tablet_service.cc:1467] Processing CreateTablet for tablet d89c384f32c847b6834084f42f755011 (DEFAULT_TABLE table=test-table [id=efa0ab688a1e4cd599e70a4e64199cba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:30.129575 12428 tablet_service.cc:1467] Processing CreateTablet for tablet d89c384f32c847b6834084f42f755011 (DEFAULT_TABLE table=test-table [id=efa0ab688a1e4cd599e70a4e64199cba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:30.129914 12614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89c384f32c847b6834084f42f755011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:30.129966 12428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89c384f32c847b6834084f42f755011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:30.132726 12806 tablet_service.cc:1467] Processing CreateTablet for tablet d89c384f32c847b6834084f42f755011 (DEFAULT_TABLE table=test-table [id=efa0ab688a1e4cd599e70a4e64199cba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 21:00:30.133152 12806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89c384f32c847b6834084f42f755011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:30.133265 12925 tablet_bootstrap.cc:492] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: Bootstrap starting.
I20260808 21:00:30.133589 12923 tablet_bootstrap.cc:492] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b: Bootstrap starting.
I20260808 21:00:30.134042 12925 tablet_bootstrap.cc:654] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:30.134254 12923 tablet_bootstrap.cc:654] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:30.134392 12925 log.cc:824] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:30.134557 12923 log.cc:824] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:30.135279 12925 tablet_bootstrap.cc:492] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: No bootstrap required, opened a new log
I20260808 21:00:30.135329 12925 ts_tablet_manager.cc:1397] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:30.135385 12923 tablet_bootstrap.cc:492] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b: No bootstrap required, opened a new log
I20260808 21:00:30.135480 12923 ts_tablet_manager.cc:1397] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:30.136650 12929 tablet_bootstrap.cc:492] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: Bootstrap starting.
I20260808 21:00:30.137383 12929 tablet_bootstrap.cc:654] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:30.137871 12929 log.cc:824] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:30.137867 12923 raft_consensus.cc:348] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } }
I20260808 21:00:30.138080 12923 raft_consensus.cc:374] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:30.138147 12923 raft_consensus.cc:729] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccd4d478356646baa2dd8bc74c106d0b, State: Initialized, Role: FOLLOWER
I20260808 21:00:30.138298 12923 consensus_queue.cc:260] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } }
I20260808 21:00:30.138273 12925 raft_consensus.cc:348] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } }
I20260808 21:00:30.138489 12925 raft_consensus.cc:374] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:30.138545 12925 raft_consensus.cc:729] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a75e7c73b4e32ac5582846e1ed5b5, State: Initialized, Role: FOLLOWER
I20260808 21:00:30.138595 12923 ts_tablet_manager.cc:1428] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 21:00:30.138667 12925 consensus_queue.cc:260] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } }
I20260808 21:00:30.138748 12518 heartbeater.cc:503] Master 127.15.232.190:40033 was elected leader, sending a full tablet report...
I20260808 21:00:30.138787 12929 tablet_bootstrap.cc:492] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: No bootstrap required, opened a new log
I20260808 21:00:30.138850 12929 ts_tablet_manager.cc:1397] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 21:00:30.139055 12925 ts_tablet_manager.cc:1428] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 21:00:30.139534 12705 heartbeater.cc:503] Master 127.15.232.190:40033 was elected leader, sending a full tablet report...
I20260808 21:00:30.141072 12929 raft_consensus.cc:348] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } }
I20260808 21:00:30.141238 12929 raft_consensus.cc:374] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:30.141286 12929 raft_consensus.cc:729] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ed1019f50f409b8a63d46fe5b03afb, State: Initialized, Role: FOLLOWER
I20260808 21:00:30.141413 12929 consensus_queue.cc:260] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } }
I20260808 21:00:30.141709 12929 ts_tablet_manager.cc:1428] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 21:00:30.141776 12904 heartbeater.cc:503] Master 127.15.232.190:40033 was elected leader, sending a full tablet report...
W20260808 21:00:30.218050 12706 tablet.cc:2369] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 21:00:30.280195 12935 raft_consensus.cc:482] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 21:00:30.280344 12935 raft_consensus.cc:504] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } }
I20260808 21:00:30.280812 12935 leader_election.cc:288] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ed1019f50f409b8a63d46fe5b03afb (127.15.232.131:40369), ccd4d478356646baa2dd8bc74c106d0b (127.15.232.129:36315)
I20260808 21:00:30.285339 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.281154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:52369 (local address 127.15.232.131:40369)
0808 21:00:30.281207 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:30.281209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:30.281243 (+ 34us) server_negotiation.cc:407] Connection header received
0808 21:00:30.281274 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:30.281277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:30.281289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:30.281355 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:30.281554 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.282354 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.284173 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.284821 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.284931 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.284949 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:30.285218 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 21:00:30.285267 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 21:00:30.285339 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.281180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.232.130:51739 (local address 127.15.232.129:36315)
0808 21:00:30.281234 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:30.281236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:30.281448 (+ 212us) server_negotiation.cc:407] Connection header received
0808 21:00:30.281531 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:30.281532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:30.281546 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:30.281607 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 21:00:30.281790 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.282502 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.284169 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.284876 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.284942 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.284949 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:30.285217 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 21:00:30.285263 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 21:00:30.285347 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.281017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.131:40369 (local address 127.15.232.130:52369)
0808 21:00:30.281106 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:30.281123 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:30.281181 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:30.281378 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:30.281384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:30.281387 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:30.281472 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.281474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.282461 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.282462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.284040 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.284044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.284989 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.284994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.285090 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.285095 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 21:00:30.285173 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 21:00:30.285242 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":83}
I20260808 21:00:30.285354 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.281138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:36315 (local address 127.15.232.130:51739)
0808 21:00:30.281367 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:30.281384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 21:00:30.281446 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:00:30.281643 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:00:30.281643 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:00:30.281645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 21:00:30.281714 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.281715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.282577 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.282578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.284043 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:00:30.284045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.284989 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.284991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:00:30.285093 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.285095 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 21:00:30.285173 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 21:00:30.285236 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":37,"thread_start_us":122,"threads_started":1}
I20260808 21:00:30.285913 12836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89c384f32c847b6834084f42f755011" candidate_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" is_pre_election: true
I20260808 21:00:30.286098 12836 raft_consensus.cc:2454] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7a75e7c73b4e32ac5582846e1ed5b5 in term 0.
I20260808 21:00:30.286443 12554 leader_election.cc:302] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ed1019f50f409b8a63d46fe5b03afb, da7a75e7c73b4e32ac5582846e1ed5b5; no voters:
I20260808 21:00:30.286617 12456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89c384f32c847b6834084f42f755011" candidate_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccd4d478356646baa2dd8bc74c106d0b" is_pre_election: true
I20260808 21:00:30.286762 12456 raft_consensus.cc:2454] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7a75e7c73b4e32ac5582846e1ed5b5 in term 0.
I20260808 21:00:30.286751 12935 raft_consensus.cc:2781] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 21:00:30.286844 12935 raft_consensus.cc:482] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 21:00:30.286882 12935 raft_consensus.cc:3037] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:30.287752 12935 raft_consensus.cc:504] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } }
I20260808 21:00:30.287910 12935 leader_election.cc:288] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [CANDIDATE]: Term 1 election: Requested vote from peers a6ed1019f50f409b8a63d46fe5b03afb (127.15.232.131:40369), ccd4d478356646baa2dd8bc74c106d0b (127.15.232.129:36315)
I20260808 21:00:30.288132 12456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89c384f32c847b6834084f42f755011" candidate_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccd4d478356646baa2dd8bc74c106d0b"
I20260808 21:00:30.288170 12836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89c384f32c847b6834084f42f755011" candidate_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ed1019f50f409b8a63d46fe5b03afb"
I20260808 21:00:30.288235 12456 raft_consensus.cc:3037] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:30.288244 12836 raft_consensus.cc:3037] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:30.289219 12836 raft_consensus.cc:2454] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da7a75e7c73b4e32ac5582846e1ed5b5 in term 1.
I20260808 21:00:30.289219 12456 raft_consensus.cc:2454] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da7a75e7c73b4e32ac5582846e1ed5b5 in term 1.
I20260808 21:00:30.289440 12554 leader_election.cc:302] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ed1019f50f409b8a63d46fe5b03afb, da7a75e7c73b4e32ac5582846e1ed5b5; no voters:
I20260808 21:00:30.289600 12935 raft_consensus.cc:2781] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:30.289814 12935 raft_consensus.cc:686] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 1 LEADER]: Becoming Leader. State: Replica: da7a75e7c73b4e32ac5582846e1ed5b5, State: Running, Role: LEADER
I20260808 21:00:30.289927 12935 consensus_queue.cc:237] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } }
I20260808 21:00:30.290838 12258 catalog_manager.cc:5495] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to da7a75e7c73b4e32ac5582846e1ed5b5 (127.15.232.130). New cstate: current_term: 1 leader_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a75e7c73b4e32ac5582846e1ed5b5" member_type: VOTER last_known_addr { host: "127.15.232.130" port: 46531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 } health_report { overall_health: UNKNOWN } } }
I20260808 21:00:30.314873 12918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.312269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.15.232.130:46531)
0808 21:00:30.312351 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:30.312353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:30.312361 (+ 8us) server_negotiation.cc:407] Connection header received
0808 21:00:30.312414 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:30.312415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:30.312434 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:30.312523 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:00:30.312672 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.313211 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.314019 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:30.314234 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:30.314313 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:30.314497 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:00:30.314620 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:00:30.314722 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:30.314741 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 21:00:30.314761 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 21:00:30.317159 12836 raft_consensus.cc:1261] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [term 1 FOLLOWER]: Refusing update from remote peer da7a75e7c73b4e32ac5582846e1ed5b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:30.317159 12456 raft_consensus.cc:1261] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [term 1 FOLLOWER]: Refusing update from remote peer da7a75e7c73b4e32ac5582846e1ed5b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:00:30.317546 12940 consensus_queue.cc:1035] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccd4d478356646baa2dd8bc74c106d0b" member_type: VOTER last_known_addr { host: "127.15.232.129" port: 36315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:30.317678 12935 consensus_queue.cc:1035] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ed1019f50f409b8a63d46fe5b03afb" member_type: VOTER last_known_addr { host: "127.15.232.131" port: 40369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:00:30.320678 12946 mvcc.cc:205] Tried to move back new op lower bound from 7316368712975806464 to 7316368712868274176. Current Snapshot: MvccSnapshot[applied={T|T < 7316368712975806464}]
I20260808 21:00:30.320885 12947 mvcc.cc:205] Tried to move back new op lower bound from 7316368712975806464 to 7316368712868274176. Current Snapshot: MvccSnapshot[applied={T|T < 7316368712975806464}]
I20260808 21:00:30.321403 12948 mvcc.cc:205] Tried to move back new op lower bound from 7316368712975806464 to 7316368712868274176. Current Snapshot: MvccSnapshot[applied={T|T < 7316368712975806464}]
I20260808 21:00:30.323495 12259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:30.323601 12259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 21:00:30.324441 12259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 284d8e9907fc480ab17256c6ed914f33: Sending DeleteTablet for 3 replicas of tablet d89c384f32c847b6834084f42f755011
I20260808 21:00:30.324759 12614 tablet_service.cc:1514] Processing DeleteTablet for tablet d89c384f32c847b6834084f42f755011 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:30 UTC) from {username='slave'} at 127.0.0.1:34388
I20260808 21:00:30.324820 12428 tablet_service.cc:1514] Processing DeleteTablet for tablet d89c384f32c847b6834084f42f755011 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:30 UTC) from {username='slave'} at 127.0.0.1:49524
I20260808 21:00:30.324966 12806 tablet_service.cc:1514] Processing DeleteTablet for tablet d89c384f32c847b6834084f42f755011 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 21:00:30 UTC) from {username='slave'} at 127.0.0.1:33584
I20260808 21:00:30.325073 12963 tablet_replica.cc:330] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b: stopping tablet replica
I20260808 21:00:30.325189 12963 raft_consensus.cc:2229] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:30.325309 12963 raft_consensus.cc:2258] T d89c384f32c847b6834084f42f755011 P ccd4d478356646baa2dd8bc74c106d0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:30.325477 12960 tablet_replica.cc:330] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: stopping tablet replica
I20260808 21:00:30.325498 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 12340
I20260808 21:00:30.325609 12960 raft_consensus.cc:2229] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 21:00:30.325717 12960 raft_consensus.cc:2258] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:30.326184 12960 ts_tablet_manager.cc:1905] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:30.326350 12964 tablet_replica.cc:330] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: stopping tablet replica
I20260808 21:00:30.326483 12964 raft_consensus.cc:2229] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 21:00:30.326619 12964 raft_consensus.cc:2258] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 21:00:30.328282 12960 ts_tablet_manager.cc:1918] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:30.328385 12960 log.cc:1196] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-1/wal/wals/d89c384f32c847b6834084f42f755011
I20260808 21:00:30.328656 12960 ts_tablet_manager.cc:1939] T d89c384f32c847b6834084f42f755011 P da7a75e7c73b4e32ac5582846e1ed5b5: Deleting consensus metadata
I20260808 21:00:30.329160 12236 catalog_manager.cc:4841] TS da7a75e7c73b4e32ac5582846e1ed5b5 (127.15.232.130:46531): tablet d89c384f32c847b6834084f42f755011 (table test-table [id=efa0ab688a1e4cd599e70a4e64199cba]) successfully deleted
I20260808 21:00:30.330410 12964 ts_tablet_manager.cc:1905] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 21:00:30.332267 12964 ts_tablet_manager.cc:1918] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 21:00:30.332355 12964 log.cc:1196] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: Deleting WAL directory at /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786222573557010-16290-0/minicluster-data/ts-2/wal/wals/d89c384f32c847b6834084f42f755011
I20260808 21:00:30.332607 12964 ts_tablet_manager.cc:1939] T d89c384f32c847b6834084f42f755011 P a6ed1019f50f409b8a63d46fe5b03afb: Deleting consensus metadata
I20260808 21:00:30.333081 12240 catalog_manager.cc:4841] TS a6ed1019f50f409b8a63d46fe5b03afb (127.15.232.131:40369): tablet d89c384f32c847b6834084f42f755011 (table test-table [id=efa0ab688a1e4cd599e70a4e64199cba]) successfully deleted
W20260808 21:00:30.345579 12237 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.232.129:36315 (error 108)
I20260808 21:00:30.346240 12915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:30.346054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.129:36315 (local address 127.0.0.1:49530)
0808 21:00:30.346118 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 21:00:30.346198 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.129:36315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 21:00:30.346357 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 12521
W20260808 21:00:30.346388 12237 catalog_manager.cc:4584] TS ccd4d478356646baa2dd8bc74c106d0b (127.15.232.129:36315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d89c384f32c847b6834084f42f755011: Network error: Client connection negotiation failed: client connection to 127.15.232.129:36315: connect: Connection refused (error 111)
I20260808 21:00:30.359177 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 12708
I20260808 21:00:30.372921 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 12218
2026-08-08T21:00:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (889 ms)
[----------] 6 tests from EncryptionPolicyTest (10289 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:30 dist-test-slave-c12x krb5kdc[12972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:30 dist-test-slave-c12x krb5kdc[12972](info): set up 2 sockets
Aug 08 21:00:30 dist-test-slave-c12x krb5kdc[12972](info): commencing operation
krb5kdc: starting...
W20260808 21:00:31.620059 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:31 dist-test-slave-c12x krb5kdc[12972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:31.759284 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33927
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:31.848871 12986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:31.849200 12986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:31.849249 12986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:31.850557 12986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:31.850644 12986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:31.850670 12986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:31.850692 12986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:31.850724 12986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:31.852432 12986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33927
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:31.852768 12986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:31.853026 12986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:31.858922 12995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:31.858855 12998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:31.859086 12986 server_base.cc:1034] running on GCE node
W20260808 21:00:31.858872 12994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:31.859661 12986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:31.860126 12986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:31.861375 12986 hybrid_clock.cc:648] HybridClock initialized: now 1786222831861292 us; error 74 us; skew 500 ppm
Aug 08 21:00:31 dist-test-slave-c12x krb5kdc[12972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222831, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:31.864346 12986 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:31.864912 12986 webserver.cc:460] Webserver started at http://127.0.0.1:45739/ using document root <none> and password file <none>
I20260808 21:00:31.865152 12986 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:31.865226 12986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:31.865391 12986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:31.866573 12986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "98d561d92ae94e7fa33cb2dfd53dd403"
format_stamp: "Formatted at 2026-08-08 21:00:31 on dist-test-slave-c12x"
server_key: "f7a082fbc7ed23ee72b6af2368d881de"
server_key_iv: "43ac2d35086e0070f634184e2eddcfb7"
server_key_version: "encryptionkey@0"
I20260808 21:00:31.866964 12986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "98d561d92ae94e7fa33cb2dfd53dd403"
format_stamp: "Formatted at 2026-08-08 21:00:31 on dist-test-slave-c12x"
server_key: "f7a082fbc7ed23ee72b6af2368d881de"
server_key_iv: "43ac2d35086e0070f634184e2eddcfb7"
server_key_version: "encryptionkey@0"
I20260808 21:00:31.868558 12986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 21:00:31.869540 13010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:31.869944 12986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:31.870078 12986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "98d561d92ae94e7fa33cb2dfd53dd403"
format_stamp: "Formatted at 2026-08-08 21:00:31 on dist-test-slave-c12x"
server_key: "f7a082fbc7ed23ee72b6af2368d881de"
server_key_iv: "43ac2d35086e0070f634184e2eddcfb7"
server_key_version: "encryptionkey@0"
I20260808 21:00:31.870165 12986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:31.881064 12986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:31.883896 12986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:31.884133 12986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:31.889063 12986 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33927
I20260808 21:00:31.889113 13091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33927 every 8 connection(s)
I20260808 21:00:31.889546 12986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:31.890589 13093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:31.893321 13093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403: Bootstrap starting.
I20260808 21:00:31.894071 13093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:31.894418 13093 log.cc:824] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:31.895371 13093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403: No bootstrap required, opened a new log
I20260808 21:00:31.896366 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 12986
I20260808 21:00:31.896513 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:31.896741 16290 external_mini_cluster.cc:1428] Setting key dd8aa8d1edc709c4589c850942f2abf4
I20260808 21:00:31.898008 13093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d561d92ae94e7fa33cb2dfd53dd403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33927 } }
I20260808 21:00:31.898204 13093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:31.898279 13093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d561d92ae94e7fa33cb2dfd53dd403, State: Initialized, Role: FOLLOWER
I20260808 21:00:31.898491 13093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d561d92ae94e7fa33cb2dfd53dd403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33927 } }
I20260808 21:00:31.898589 13093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:31.898617 13093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:31.898687 13093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:31.899945 13093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d561d92ae94e7fa33cb2dfd53dd403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33927 } }
I20260808 21:00:31.900098 13093 leader_election.cc:302] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98d561d92ae94e7fa33cb2dfd53dd403; no voters:
I20260808 21:00:31.900305 13093 leader_election.cc:288] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:31.900363 13099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:31.900538 13099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [term 1 LEADER]: Becoming Leader. State: Replica: 98d561d92ae94e7fa33cb2dfd53dd403, State: Running, Role: LEADER
I20260808 21:00:31.900621 13093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:31.900693 13099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d561d92ae94e7fa33cb2dfd53dd403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33927 } }
I20260808 21:00:31.901986 13102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98d561d92ae94e7fa33cb2dfd53dd403. Latest consensus state: current_term: 1 leader_uuid: "98d561d92ae94e7fa33cb2dfd53dd403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d561d92ae94e7fa33cb2dfd53dd403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33927 } } }
I20260808 21:00:31.902124 13102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:31.902156 13104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98d561d92ae94e7fa33cb2dfd53dd403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d561d92ae94e7fa33cb2dfd53dd403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33927 } } }
I20260808 21:00:31.902245 13104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:31.902333 13119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:31.902890 13119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:31.905656 13119 catalog_manager.cc:1347] Generated new cluster ID: e9f3265ed3f24e0baaec2bd53952b7cb
I20260808 21:00:31.905725 13119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 21:00:31 dist-test-slave-c12x krb5kdc[12972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 21:00:31.912348 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:31.897401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44417 (local address 127.0.0.1:33927)
0808 21:00:31.897635 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:31.897644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:31.897672 (+ 28us) server_negotiation.cc:407] Connection header received
0808 21:00:31.898002 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:31.898017 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:31.898086 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:31.898260 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:31.899644 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:31.900947 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:31.901871 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:31.902530 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:31.902674 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:31.908468 (+ 5794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:31.908475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:31.908476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:31.908495 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:31.910512 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:31.911027 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:31.911031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:31.911031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:31.911142 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:31.911399 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:31.911400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:31.911401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:31.911729 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:31.911931 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:31.912036 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 21:00:31.912151 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":116,"threads_started":1}
I20260808 21:00:31.913723 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:31.920219 13119 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:31.920778 13119 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:31.925033 13119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98d561d92ae94e7fa33cb2dfd53dd403: Generated new TSK 0
I20260808 21:00:31.925253 13119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:31.931604 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:31.915259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54975 (local address 127.0.0.1:33927)
0808 21:00:31.915327 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:31.915329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:31.919680 (+ 4351us) server_negotiation.cc:407] Connection header received
0808 21:00:31.919760 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:31.919761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:31.919772 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:31.919821 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:31.925090 (+ 5269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:31.925687 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:31.926449 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:31.926687 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:31.926797 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:31.928401 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:31.928403 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:31.928404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:31.928412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:31.930118 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:31.930607 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:31.930608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:31.930608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:31.930679 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:31.930961 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:31.930962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:31.930962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:31.931101 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:31.931232 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:31.931381 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 21:00:31.931484 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 21:00:31.934027 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 12986
W20260808 21:00:31.941500 12966 connection.cc:535] client connection to 127.0.0.1:33927 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33927): Connection reset by peer (error 104)
2026-08-08T21:00:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1555 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:31 dist-test-slave-c12x krb5kdc[13139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:32 dist-test-slave-c12x krb5kdc[13139](info): set up 2 sockets
Aug 08 21:00:32 dist-test-slave-c12x krb5kdc[13139](info): commencing operation
krb5kdc: starting...
W20260808 21:00:33.243865 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:33 dist-test-slave-c12x krb5kdc[13139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:33.390374 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:33.478018 13159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:33.478303 13159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:33.478349 13159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:33.479646 13159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:33.479720 13159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:33.479745 13159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:33.479774 13159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:33.479807 13159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:33.481436 13159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:33.481760 13159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:33.482026 13159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:33.487393 13171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:33.487512 13167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:33.487393 13166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:33.487726 13159 server_base.cc:1034] running on GCE node
I20260808 21:00:33.488044 13159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:33.488396 13159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:33.489598 13159 hybrid_clock.cc:648] HybridClock initialized: now 1786222833489599 us; error 54 us; skew 500 ppm
Aug 08 21:00:33 dist-test-slave-c12x krb5kdc[13139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222833, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:33.499172 13159 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:33.500056 13159 webserver.cc:460] Webserver started at http://127.0.0.1:39235/ using document root <none> and password file <none>
I20260808 21:00:33.500582 13159 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:33.500721 13159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:33.500903 13159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:33.502108 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "a58c21f8874848f38ac3236a3382b2f5"
format_stamp: "Formatted at 2026-08-08 21:00:33 on dist-test-slave-c12x"
server_key: "c1d843a06840f07204fd3e3e4ef01187"
server_key_iv: "cafb5005f282fd126e6fce265c9bd388"
server_key_version: "encryptionkey@0"
I20260808 21:00:33.502525 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "a58c21f8874848f38ac3236a3382b2f5"
format_stamp: "Formatted at 2026-08-08 21:00:33 on dist-test-slave-c12x"
server_key: "c1d843a06840f07204fd3e3e4ef01187"
server_key_iv: "cafb5005f282fd126e6fce265c9bd388"
server_key_version: "encryptionkey@0"
I20260808 21:00:33.504150 13159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 21:00:33.505086 13179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:33.505412 13159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:33.505573 13159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "a58c21f8874848f38ac3236a3382b2f5"
format_stamp: "Formatted at 2026-08-08 21:00:33 on dist-test-slave-c12x"
server_key: "c1d843a06840f07204fd3e3e4ef01187"
server_key_iv: "cafb5005f282fd126e6fce265c9bd388"
server_key_version: "encryptionkey@0"
I20260808 21:00:33.505671 13159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:33.526607 13159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:33.529373 13159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:33.529575 13159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:33.534431 13159 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43527
I20260808 21:00:33.534485 13250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43527 every 8 connection(s)
I20260808 21:00:33.534902 13159 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:33.535895 13251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:33.537281 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 13159
I20260808 21:00:33.537439 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:33.537652 16290 external_mini_cluster.cc:1428] Setting key ebf2698a426ada582ed7141464da3bad
I20260808 21:00:33.538859 13251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5: Bootstrap starting.
I20260808 21:00:33.539712 13251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:33.540026 13251 log.cc:824] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:33.540920 13251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5: No bootstrap required, opened a new log
I20260808 21:00:33.543562 13251 raft_consensus.cc:348] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58c21f8874848f38ac3236a3382b2f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43527 } }
I20260808 21:00:33.543735 13251 raft_consensus.cc:374] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:33.543782 13251 raft_consensus.cc:729] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58c21f8874848f38ac3236a3382b2f5, State: Initialized, Role: FOLLOWER
I20260808 21:00:33.543924 13251 consensus_queue.cc:260] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58c21f8874848f38ac3236a3382b2f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43527 } }
I20260808 21:00:33.544039 13251 raft_consensus.cc:388] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:33.544097 13251 raft_consensus.cc:482] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:33.544180 13251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:33.544312 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:33.538258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45447 (local address 127.0.0.1:43527)
0808 21:00:33.538451 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:33.538460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:33.538486 (+ 26us) server_negotiation.cc:407] Connection header received
0808 21:00:33.538926 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:33.538933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:33.539009 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:33.539152 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:33.539896 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:33.540892 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:33.542030 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:33.542477 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:33.542569 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:33.543358 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:33.543362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:33.543362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:33.543621 (+ 259us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:33.543656 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:33.543668 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:33.543773 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:33.543937 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 21:00:33.543973 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260808 21:00:33.544940 13251 raft_consensus.cc:504] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58c21f8874848f38ac3236a3382b2f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43527 } }
I20260808 21:00:33.545071 13251 leader_election.cc:302] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a58c21f8874848f38ac3236a3382b2f5; no voters:
I20260808 21:00:33.545276 13251 leader_election.cc:288] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:33.545281 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:33.545305 13259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:33.545470 13259 raft_consensus.cc:686] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [term 1 LEADER]: Becoming Leader. State: Replica: a58c21f8874848f38ac3236a3382b2f5, State: Running, Role: LEADER
I20260808 21:00:33.545540 13251 sys_catalog.cc:564] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:33.545583 13259 consensus_queue.cc:237] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58c21f8874848f38ac3236a3382b2f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43527 } }
I20260808 21:00:33.545888 13259 sys_catalog.cc:455] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a58c21f8874848f38ac3236a3382b2f5. Latest consensus state: current_term: 1 leader_uuid: "a58c21f8874848f38ac3236a3382b2f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58c21f8874848f38ac3236a3382b2f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43527 } } }
I20260808 21:00:33.546008 13259 sys_catalog.cc:458] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:33.545907 13264 sys_catalog.cc:455] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a58c21f8874848f38ac3236a3382b2f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58c21f8874848f38ac3236a3382b2f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43527 } } }
I20260808 21:00:33.546156 13264 sys_catalog.cc:458] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:33.546300 13269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:33.546871 13269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:33.550508 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:33.547015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38287 (local address 127.0.0.1:43527)
0808 21:00:33.547054 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:33.547057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:33.547092 (+ 35us) server_negotiation.cc:407] Connection header received
0808 21:00:33.547201 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:33.547202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:33.547226 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:33.547263 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:33.547608 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:33.548338 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:33.549090 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:33.549327 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:33.549412 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:33.550183 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:33.550184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:33.550184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:33.550196 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:33.550210 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:33.550214 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:33.550283 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:33.550385 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 21:00:33.550397 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 21:00:33.551919 13269 catalog_manager.cc:1347] Generated new cluster ID: f062d70f005c4ddb97111cf97017e033
I20260808 21:00:33.551988 13269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:33.559648 13269 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:33.560217 13269 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:33.562481 13269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a58c21f8874848f38ac3236a3382b2f5: Generated new TSK 0
I20260808 21:00:33.562691 13269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:33.566910 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 13159
W20260808 21:00:33.577787 13137 connection.cc:535] client connection to 127.0.0.1:43527 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43527): Connection reset by peer (error 104)
2026-08-08T21:00:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1636 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:33 dist-test-slave-c12x krb5kdc[13295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:33 dist-test-slave-c12x krb5kdc[13295](info): set up 2 sockets
Aug 08 21:00:33 dist-test-slave-c12x krb5kdc[13295](info): commencing operation
krb5kdc: starting...
W20260808 21:00:34.804292 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:34 dist-test-slave-c12x krb5kdc[13295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:34.943022 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:35.034260 13314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:35.034582 13314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:35.034631 13314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:35.036092 13314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:35.036181 13314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:35.036208 13314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:35.036242 13314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:35.036278 13314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:35.038084 13314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:35.038440 13314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:35.038718 13314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:35.044319 13320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:35.044363 13319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:35.044308 13322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:35.044579 13314 server_base.cc:1034] running on GCE node
I20260808 21:00:35.044926 13314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:35.045275 13314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:35.046464 13314 hybrid_clock.cc:648] HybridClock initialized: now 1786222835046458 us; error 46 us; skew 500 ppm
Aug 08 21:00:35 dist-test-slave-c12x krb5kdc[13295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222835, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:35.049572 13314 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:35.050104 13314 webserver.cc:460] Webserver started at http://127.0.0.1:34121/ using document root <none> and password file <none>
I20260808 21:00:35.050369 13314 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:35.050449 13314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:35.050621 13314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:35.051978 13314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "03b2a4099213450983863c3ff34724ef"
format_stamp: "Formatted at 2026-08-08 21:00:35 on dist-test-slave-c12x"
server_key: "c0fb9f931bc3e66fcaa6b2eaf6a6903a"
server_key_iv: "88a5f33715668ae0ff99d648fc89cf62"
server_key_version: "encryptionkey@0"
I20260808 21:00:35.052412 13314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "03b2a4099213450983863c3ff34724ef"
format_stamp: "Formatted at 2026-08-08 21:00:35 on dist-test-slave-c12x"
server_key: "c0fb9f931bc3e66fcaa6b2eaf6a6903a"
server_key_iv: "88a5f33715668ae0ff99d648fc89cf62"
server_key_version: "encryptionkey@0"
I20260808 21:00:35.054010 13314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:00:35.054926 13336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:35.055173 13314 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:35.055296 13314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "03b2a4099213450983863c3ff34724ef"
format_stamp: "Formatted at 2026-08-08 21:00:35 on dist-test-slave-c12x"
server_key: "c0fb9f931bc3e66fcaa6b2eaf6a6903a"
server_key_iv: "88a5f33715668ae0ff99d648fc89cf62"
server_key_version: "encryptionkey@0"
I20260808 21:00:35.055392 13314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:35.059427 13314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:35.061906 13314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:35.062033 13314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:35.066690 13314 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39807
I20260808 21:00:35.066747 13418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39807 every 8 connection(s)
I20260808 21:00:35.067152 13314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:35.068358 13420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:35.069638 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 13314
I20260808 21:00:35.069826 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:35.070022 16290 external_mini_cluster.cc:1428] Setting key ead1b5b931e9cc45e08c98c0dc8cba10
I20260808 21:00:35.071321 13420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef: Bootstrap starting.
I20260808 21:00:35.072242 13420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:35.072626 13420 log.cc:824] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:35.072857 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:35.070668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34663 (local address 127.0.0.1:39807)
0808 21:00:35.070880 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:35.070888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:35.070913 (+ 25us) server_negotiation.cc:407] Connection header received
0808 21:00:35.071356 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:35.071363 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:35.071389 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:35.071584 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:35.072298 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:35.072302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:35.072302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:35.072306 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:35.072375 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:35.072397 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:35.072468 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:35.072553 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 21:00:35.072614 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260808 21:00:35.073493 13420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef: No bootstrap required, opened a new log
I20260808 21:00:35.074159 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:35.075718 13420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b2a4099213450983863c3ff34724ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39807 } }
I20260808 21:00:35.075958 13420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:35.076026 13420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b2a4099213450983863c3ff34724ef, State: Initialized, Role: FOLLOWER
I20260808 21:00:35.076174 13420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b2a4099213450983863c3ff34724ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39807 } }
I20260808 21:00:35.076292 13420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:35.076339 13420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:35.076390 13420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:35.077103 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:35.075905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56217 (local address 127.0.0.1:39807)
0808 21:00:35.075951 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:35.075953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:35.076002 (+ 49us) server_negotiation.cc:407] Connection header received
0808 21:00:35.076093 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:35.076094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:35.076100 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:35.076169 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:35.076853 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:35.076854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:35.076855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:35.076860 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:35.076875 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:35.076879 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:35.076944 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:35.077000 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 21:00:35.077017 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 21:00:35.077175 13420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b2a4099213450983863c3ff34724ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39807 } }
I20260808 21:00:35.077278 13420 leader_election.cc:302] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03b2a4099213450983863c3ff34724ef; no voters:
I20260808 21:00:35.077448 13420 leader_election.cc:288] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:35.077483 13433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:35.077648 13433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [term 1 LEADER]: Becoming Leader. State: Replica: 03b2a4099213450983863c3ff34724ef, State: Running, Role: LEADER
I20260808 21:00:35.077747 13420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:35.077756 13433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b2a4099213450983863c3ff34724ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39807 } }
I20260808 21:00:35.078183 13434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03b2a4099213450983863c3ff34724ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b2a4099213450983863c3ff34724ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39807 } } }
I20260808 21:00:35.078229 13435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03b2a4099213450983863c3ff34724ef. Latest consensus state: current_term: 1 leader_uuid: "03b2a4099213450983863c3ff34724ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b2a4099213450983863c3ff34724ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39807 } } }
I20260808 21:00:35.078342 13434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:35.078398 13435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef [sys.catalog]: This master's current role is: LEADER
W20260808 21:00:35.079713 13461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:00:35.079762 13461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 21:00:35.079841 13444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:35.079998 13444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:35.081629 13444 catalog_manager.cc:1347] Generated new cluster ID: 9777619bac0e4f80bfc9115b03db80d2
I20260808 21:00:35.081679 13444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:35.088378 13444 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:35.088614 13444 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:35.098449 13444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03b2a4099213450983863c3ff34724ef: Generated new TSK 0
I20260808 21:00:35.098716 13444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:35.105829 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 13314
2026-08-08T21:00:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:35 dist-test-slave-c12x krb5kdc[13471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:35 dist-test-slave-c12x krb5kdc[13471](info): set up 2 sockets
Aug 08 21:00:35 dist-test-slave-c12x krb5kdc[13471](info): commencing operation
krb5kdc: starting...
W20260808 21:00:36.388793 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:36 dist-test-slave-c12x krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:36.527029 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:36.614374 13492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:36.614651 13492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:36.614692 13492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:36.615923 13492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:36.615970 13492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:36.615985 13492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:36.615998 13492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:36.616009 13492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:36.617599 13492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:36.617887 13492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:36.618127 13492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:36.623651 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:36.623617 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:36.623641 13497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:36.624068 13492 server_base.cc:1034] running on GCE node
I20260808 21:00:36.624333 13492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:36.624646 13492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:36.625854 13492 hybrid_clock.cc:648] HybridClock initialized: now 1786222836625824 us; error 36 us; skew 500 ppm
Aug 08 21:00:36 dist-test-slave-c12x krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222836, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:36.628784 13492 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:36.629328 13492 webserver.cc:460] Webserver started at http://127.0.0.1:41727/ using document root <none> and password file <none>
I20260808 21:00:36.629598 13492 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:36.629683 13492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:36.629846 13492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:36.631024 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "3b6d2683c104443cb20bd2b13c827b24"
format_stamp: "Formatted at 2026-08-08 21:00:36 on dist-test-slave-c12x"
server_key: "992838b193aa71ab8e64e09056f3b88c"
server_key_iv: "3c3efbefc2059c961d6dd3269cb529cf"
server_key_version: "encryptionkey@0"
I20260808 21:00:36.631428 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "3b6d2683c104443cb20bd2b13c827b24"
format_stamp: "Formatted at 2026-08-08 21:00:36 on dist-test-slave-c12x"
server_key: "992838b193aa71ab8e64e09056f3b88c"
server_key_iv: "3c3efbefc2059c961d6dd3269cb529cf"
server_key_version: "encryptionkey@0"
I20260808 21:00:36.632959 13492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:36.633872 13515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:36.634179 13492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:36.634352 13492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "3b6d2683c104443cb20bd2b13c827b24"
format_stamp: "Formatted at 2026-08-08 21:00:36 on dist-test-slave-c12x"
server_key: "992838b193aa71ab8e64e09056f3b88c"
server_key_iv: "3c3efbefc2059c961d6dd3269cb529cf"
server_key_version: "encryptionkey@0"
I20260808 21:00:36.634449 13492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:36.669464 13492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:36.672020 13492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:36.672155 13492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:36.676472 13492 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40967
I20260808 21:00:36.676518 13589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40967 every 8 connection(s)
I20260808 21:00:36.676939 13492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:36.677953 13590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:36.680545 13590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24: Bootstrap starting.
I20260808 21:00:36.681190 13590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:36.681535 13590 log.cc:824] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:36.682344 13590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24: No bootstrap required, opened a new log
I20260808 21:00:36.684167 13590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6d2683c104443cb20bd2b13c827b24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } }
I20260808 21:00:36.684322 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 13492
I20260808 21:00:36.684367 13590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:36.684424 13590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b6d2683c104443cb20bd2b13c827b24, State: Initialized, Role: FOLLOWER
I20260808 21:00:36.684532 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:36.684631 13590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6d2683c104443cb20bd2b13c827b24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } }
I20260808 21:00:36.684751 13590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:36.684790 16290 external_mini_cluster.cc:1428] Setting key b302129bb9805b81a44ecaba7cd992a6
I20260808 21:00:36.684801 13590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:36.684854 13590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:36.686147 13590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6d2683c104443cb20bd2b13c827b24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } }
I20260808 21:00:36.686270 13590 leader_election.cc:302] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b6d2683c104443cb20bd2b13c827b24; no voters:
I20260808 21:00:36.686475 13590 leader_election.cc:288] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:36.686506 13596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:36.686626 13596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [term 1 LEADER]: Becoming Leader. State: Replica: 3b6d2683c104443cb20bd2b13c827b24, State: Running, Role: LEADER
I20260808 21:00:36.686765 13590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:36.686775 13596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6d2683c104443cb20bd2b13c827b24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } }
I20260808 21:00:36.687247 13596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b6d2683c104443cb20bd2b13c827b24. Latest consensus state: current_term: 1 leader_uuid: "3b6d2683c104443cb20bd2b13c827b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6d2683c104443cb20bd2b13c827b24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } } }
I20260808 21:00:36.687348 13596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:36.687629 13615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:36.688261 13615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:36.688344 13600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b6d2683c104443cb20bd2b13c827b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b6d2683c104443cb20bd2b13c827b24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40967 } } }
I20260808 21:00:36.688524 13600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:36.691461 13615 catalog_manager.cc:1347] Generated new cluster ID: 2a7e79775e3e4e1ca39e1225e24645d7
I20260808 21:00:36.691560 13615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 21:00:36 dist-test-slave-c12x krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 21:00:36.696671 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:36.685375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37463 (local address 127.0.0.1:40967)
0808 21:00:36.685723 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:36.685737 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:36.685785 (+ 48us) server_negotiation.cc:407] Connection header received
0808 21:00:36.685913 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:36.685921 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:36.685981 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:36.686139 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:36.687069 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:36.688638 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:36.689698 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:36.690305 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:36.690435 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:36.693090 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:36.693094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:36.693094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:36.693107 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:36.694902 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:36.695357 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:36.695360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:36.695361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:36.695536 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:36.695783 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:36.695784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:36.695785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:36.696058 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:36.696243 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:36.696318 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 21:00:36.696480 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":177,"threads_started":1}
I20260808 21:00:36.697950 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:36.702378 13615 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:36.704025 13615 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:36.710321 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:36.699690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57049 (local address 127.0.0.1:40967)
0808 21:00:36.699732 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:36.699734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:36.703663 (+ 3929us) server_negotiation.cc:407] Connection header received
0808 21:00:36.703758 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:36.703759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:36.704028 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:36.704115 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:36.704747 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:36.705251 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:36.706025 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:36.706253 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:36.706329 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:36.707680 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:36.707681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:36.707681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:36.707688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:36.709144 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:36.709518 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:36.709520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:36.709520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:36.709584 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:36.709791 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:36.709792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:36.709792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:36.709907 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:36.710018 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:36.710121 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 21:00:36.710211 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 21:00:36.734669 13615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b6d2683c104443cb20bd2b13c827b24: Generated new TSK 0
I20260808 21:00:36.734925 13615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:36.739627 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 13492
2026-08-08T21:00:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1626 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:36 dist-test-slave-c12x krb5kdc[13638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:36 dist-test-slave-c12x krb5kdc[13638](info): set up 2 sockets
Aug 08 21:00:36 dist-test-slave-c12x krb5kdc[13638](info): commencing operation
krb5kdc: starting...
W20260808 21:00:38.082685 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:38 dist-test-slave-c12x krb5kdc[13638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:38.223498 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:38.318171 13654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:38.318478 13654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:38.318522 13654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:38.320000 13654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:38.320112 13654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:38.320140 13654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:38.320174 13654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:38.320197 13654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:38.322186 13654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:38.322553 13654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:38.322871 13654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:38.328341 13667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:38.328353 13664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:38.328475 13662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:38.328780 13654 server_base.cc:1034] running on GCE node
I20260808 21:00:38.329063 13654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:38.329401 13654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:38.330585 13654 hybrid_clock.cc:648] HybridClock initialized: now 1786222838330557 us; error 45 us; skew 500 ppm
Aug 08 21:00:38 dist-test-slave-c12x krb5kdc[13638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:38.333617 13654 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:38.334177 13654 webserver.cc:460] Webserver started at http://127.0.0.1:37261/ using document root <none> and password file <none>
I20260808 21:00:38.334465 13654 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:38.334555 13654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:38.334733 13654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:38.336108 13654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "2db41d2786c146a28d130030bacd4cf8"
format_stamp: "Formatted at 2026-08-08 21:00:38 on dist-test-slave-c12x"
server_key: "582f62da444e88a1d93ff4bc7955e874"
server_key_iv: "115585af6976c809b4c32d1a9f3b7471"
server_key_version: "encryptionkey@0"
I20260808 21:00:38.336544 13654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "2db41d2786c146a28d130030bacd4cf8"
format_stamp: "Formatted at 2026-08-08 21:00:38 on dist-test-slave-c12x"
server_key: "582f62da444e88a1d93ff4bc7955e874"
server_key_iv: "115585af6976c809b4c32d1a9f3b7471"
server_key_version: "encryptionkey@0"
I20260808 21:00:38.338224 13654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:00:38.339223 13678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:38.339524 13654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:38.339653 13654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "2db41d2786c146a28d130030bacd4cf8"
format_stamp: "Formatted at 2026-08-08 21:00:38 on dist-test-slave-c12x"
server_key: "582f62da444e88a1d93ff4bc7955e874"
server_key_iv: "115585af6976c809b4c32d1a9f3b7471"
server_key_version: "encryptionkey@0"
I20260808 21:00:38.339757 13654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:38.353272 13654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:38.355851 13654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:38.356052 13654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:38.360841 13654 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36977
I20260808 21:00:38.360948 13757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36977 every 8 connection(s)
I20260808 21:00:38.361366 13654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:38.362417 13758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:38.365067 13758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8: Bootstrap starting.
I20260808 21:00:38.365677 13758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:38.365949 13758 log.cc:824] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:38.366761 13758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8: No bootstrap required, opened a new log
I20260808 21:00:38.368575 13758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db41d2786c146a28d130030bacd4cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260808 21:00:38.368748 13758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:38.368773 13758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db41d2786c146a28d130030bacd4cf8, State: Initialized, Role: FOLLOWER
I20260808 21:00:38.368975 13758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db41d2786c146a28d130030bacd4cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260808 21:00:38.369046 13758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:38.369068 13758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:38.369119 13758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:38.369683 13758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db41d2786c146a28d130030bacd4cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260808 21:00:38.369778 13758 leader_election.cc:302] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2db41d2786c146a28d130030bacd4cf8; no voters:
I20260808 21:00:38.369976 13758 leader_election.cc:288] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:38.370004 13762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:38.370222 13762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [term 1 LEADER]: Becoming Leader. State: Replica: 2db41d2786c146a28d130030bacd4cf8, State: Running, Role: LEADER
I20260808 21:00:38.370321 13758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:38.370406 13762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db41d2786c146a28d130030bacd4cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } }
I20260808 21:00:38.370489 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 13654
I20260808 21:00:38.370622 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:38.370822 16290 external_mini_cluster.cc:1428] Setting key 720548f06e64a28bf315de96537fc25e
I20260808 21:00:38.370791 13762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db41d2786c146a28d130030bacd4cf8. Latest consensus state: current_term: 1 leader_uuid: "2db41d2786c146a28d130030bacd4cf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db41d2786c146a28d130030bacd4cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } } }
I20260808 21:00:38.370863 13762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:38.370887 13764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2db41d2786c146a28d130030bacd4cf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db41d2786c146a28d130030bacd4cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36977 } } }
I20260808 21:00:38.370945 13764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:38.371460 13768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:38.372064 13768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:38.373832 13768 catalog_manager.cc:1347] Generated new cluster ID: 5260668bb55a49229f913beddc638afb
I20260808 21:00:38.373890 13768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:38.377782 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:38.371548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56197 (local address 127.0.0.1:36977)
0808 21:00:38.372066 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:38.372077 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:38.372129 (+ 52us) server_negotiation.cc:407] Connection header received
0808 21:00:38.372260 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:38.372269 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:38.372327 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:38.372469 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:38.372736 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:38.374131 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:38.375396 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:38.376110 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:38.376235 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:38.376998 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:38.377002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:38.377003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:38.377243 (+ 240us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:38.377298 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:38.377315 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:38.377390 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:38.377466 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 21:00:38.377537 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":71,"threads_started":1}
I20260808 21:00:38.378818 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:38.384117 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:38.380384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55839 (local address 127.0.0.1:36977)
0808 21:00:38.380420 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:38.380422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:38.380559 (+ 137us) server_negotiation.cc:407] Connection header received
0808 21:00:38.380621 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:38.380622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:38.380632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:38.380690 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:38.380825 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:38.381567 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:38.382627 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:38.382927 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:38.383006 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:38.383831 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:38.383832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:38.383833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:38.383839 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:38.383856 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:38.383862 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:38.383920 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:38.384003 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 21:00:38.384029 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 21:00:38.384202 13768 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:38.384923 13768 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:38.388499 13768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2db41d2786c146a28d130030bacd4cf8: Generated new TSK 0
I20260808 21:00:38.388746 13768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:38.390820 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 13654
2026-08-08T21:00:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1647 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:38 dist-test-slave-c12x krb5kdc[13801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:38 dist-test-slave-c12x krb5kdc[13801](info): set up 2 sockets
Aug 08 21:00:38 dist-test-slave-c12x krb5kdc[13801](info): commencing operation
krb5kdc: starting...
W20260808 21:00:39.716210 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:39 dist-test-slave-c12x krb5kdc[13801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:39.858163 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36151
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:39.946200 13821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:39.946473 13821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:39.946520 13821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:39.947796 13821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:39.947878 13821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:39.947904 13821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:39.947928 13821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:39.947956 13821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:39.949646 13821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:39.949976 13821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:39.950242 13821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:39.955777 13831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:39.955776 13833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:39.955772 13836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:39.956136 13821 server_base.cc:1034] running on GCE node
I20260808 21:00:39.956439 13821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:39.956714 13821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:39.957887 13821 hybrid_clock.cc:648] HybridClock initialized: now 1786222839957865 us; error 42 us; skew 500 ppm
Aug 08 21:00:39 dist-test-slave-c12x krb5kdc[13801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:39.966677 13821 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:39.967221 13821 webserver.cc:460] Webserver started at http://127.0.0.1:43859/ using document root <none> and password file <none>
I20260808 21:00:39.967569 13821 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:39.967653 13821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:39.967818 13821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:39.968976 13821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "f1e6d7bd9e3f453a980b43e66f7189ee"
format_stamp: "Formatted at 2026-08-08 21:00:39 on dist-test-slave-c12x"
server_key: "00ce1991e52f0c09fb78a8d055f7a79d"
server_key_iv: "adf2de9dbb02a158dce016a90fcac3a9"
server_key_version: "encryptionkey@0"
I20260808 21:00:39.969424 13821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "f1e6d7bd9e3f453a980b43e66f7189ee"
format_stamp: "Formatted at 2026-08-08 21:00:39 on dist-test-slave-c12x"
server_key: "00ce1991e52f0c09fb78a8d055f7a79d"
server_key_iv: "adf2de9dbb02a158dce016a90fcac3a9"
server_key_version: "encryptionkey@0"
I20260808 21:00:39.970973 13821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 21:00:39.971964 13845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:39.972353 13821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 21:00:39.972512 13821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "f1e6d7bd9e3f453a980b43e66f7189ee"
format_stamp: "Formatted at 2026-08-08 21:00:39 on dist-test-slave-c12x"
server_key: "00ce1991e52f0c09fb78a8d055f7a79d"
server_key_iv: "adf2de9dbb02a158dce016a90fcac3a9"
server_key_version: "encryptionkey@0"
I20260808 21:00:39.972612 13821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:39.976666 13821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:39.979514 13821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:39.979652 13821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:39.983731 13821 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36151
I20260808 21:00:39.983767 13930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36151 every 8 connection(s)
I20260808 21:00:39.984159 13821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:39.985159 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 13821
I20260808 21:00:39.985294 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:39.985518 16290 external_mini_cluster.cc:1428] Setting key 2ae433bbcf052623d15282fa7fdd8db7
I20260808 21:00:39.985528 13931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:39.988785 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:39.986331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47103 (local address 127.0.0.1:36151)
0808 21:00:39.986553 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:39.986561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:39.986786 (+ 225us) server_negotiation.cc:407] Connection header received
0808 21:00:39.987282 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:39.987290 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:39.987316 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:39.987595 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:39.988287 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:39.988291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:39.988291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:39.988295 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:39.988333 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:39.988344 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:39.988402 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:39.988502 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 21:00:39.988567 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":107,"threads_started":1}
I20260808 21:00:39.989511 13931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee: Bootstrap starting.
I20260808 21:00:39.989961 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:39.990351 13931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:39.990692 13931 log.cc:824] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:39.991639 13931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee: No bootstrap required, opened a new log
I20260808 21:00:39.993045 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:39.991780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52063 (local address 127.0.0.1:36151)
0808 21:00:39.991830 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:39.991837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:39.991981 (+ 144us) server_negotiation.cc:407] Connection header received
0808 21:00:39.992059 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:39.992060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:39.992064 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:39.992113 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:39.992751 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:39.992753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:39.992753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:39.992758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:39.992772 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:39.992777 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:39.992839 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:39.992922 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 21:00:39.992933 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 21:00:39.994334 13931 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6d7bd9e3f453a980b43e66f7189ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } }
I20260808 21:00:39.994518 13931 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:39.994567 13931 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e6d7bd9e3f453a980b43e66f7189ee, State: Initialized, Role: FOLLOWER
I20260808 21:00:39.994697 13931 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6d7bd9e3f453a980b43e66f7189ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } }
I20260808 21:00:39.994814 13931 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:39.994863 13931 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:39.994915 13931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:39.995640 13931 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6d7bd9e3f453a980b43e66f7189ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } }
I20260808 21:00:39.995772 13931 leader_election.cc:302] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1e6d7bd9e3f453a980b43e66f7189ee; no voters:
I20260808 21:00:39.995980 13931 leader_election.cc:288] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:39.996084 13946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:39.996281 13946 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [term 1 LEADER]: Becoming Leader. State: Replica: f1e6d7bd9e3f453a980b43e66f7189ee, State: Running, Role: LEADER
I20260808 21:00:39.996320 13931 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:39.996415 13946 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6d7bd9e3f453a980b43e66f7189ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } }
I20260808 21:00:39.996841 13947 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1e6d7bd9e3f453a980b43e66f7189ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6d7bd9e3f453a980b43e66f7189ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } } }
I20260808 21:00:39.996850 13948 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1e6d7bd9e3f453a980b43e66f7189ee. Latest consensus state: current_term: 1 leader_uuid: "f1e6d7bd9e3f453a980b43e66f7189ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6d7bd9e3f453a980b43e66f7189ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36151 } } }
I20260808 21:00:39.997071 13948 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:39.996994 13947 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:39.997350 13958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:39.998021 13958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:39.999917 13958 catalog_manager.cc:1347] Generated new cluster ID: 01dbbd178726402d9f598012c02a5902
I20260808 21:00:40.000008 13958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:40.013250 13958 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:40.013471 13958 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:40.017938 13958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1e6d7bd9e3f453a980b43e66f7189ee: Generated new TSK 0
I20260808 21:00:40.018195 13958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:40.028412 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 13821
2026-08-08T21:00:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1637 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:40 dist-test-slave-c12x krb5kdc[13973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:40 dist-test-slave-c12x krb5kdc[13973](info): set up 2 sockets
Aug 08 21:00:40 dist-test-slave-c12x krb5kdc[13973](info): commencing operation
krb5kdc: starting...
W20260808 21:00:41.285811 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:41 dist-test-slave-c12x krb5kdc[13973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:41.428390 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39439
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:41.518165 13991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:41.518441 13991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:41.518486 13991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:41.519793 13991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:41.519863 13991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:41.519891 13991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:41.519918 13991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:41.519940 13991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:41.521610 13991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:41.521917 13991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:41.522161 13991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:41.527487 13999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:41.527665 13991 server_base.cc:1034] running on GCE node
W20260808 21:00:41.527493 14002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:41.527568 14005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:41.528123 13991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:41.528482 13991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:41.529673 13991 hybrid_clock.cc:648] HybridClock initialized: now 1786222841529656 us; error 29 us; skew 500 ppm
Aug 08 21:00:41 dist-test-slave-c12x krb5kdc[13973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:41.532572 13991 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:41.533135 13991 webserver.cc:460] Webserver started at http://127.0.0.1:43547/ using document root <none> and password file <none>
I20260808 21:00:41.533409 13991 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:41.533486 13991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:41.533645 13991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:41.534788 13991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "59ef1a53d45d4f2fb05f8eca873e53a2"
format_stamp: "Formatted at 2026-08-08 21:00:41 on dist-test-slave-c12x"
server_key: "5640fbeee5706480eb665e6ef74318f0"
server_key_iv: "3b9622596aa2da6fe1fd42e88ffcf829"
server_key_version: "encryptionkey@0"
I20260808 21:00:41.535176 13991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "59ef1a53d45d4f2fb05f8eca873e53a2"
format_stamp: "Formatted at 2026-08-08 21:00:41 on dist-test-slave-c12x"
server_key: "5640fbeee5706480eb665e6ef74318f0"
server_key_iv: "3b9622596aa2da6fe1fd42e88ffcf829"
server_key_version: "encryptionkey@0"
I20260808 21:00:41.536716 13991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 21:00:41.537703 14014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:41.538009 13991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:41.538146 13991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "59ef1a53d45d4f2fb05f8eca873e53a2"
format_stamp: "Formatted at 2026-08-08 21:00:41 on dist-test-slave-c12x"
server_key: "5640fbeee5706480eb665e6ef74318f0"
server_key_iv: "3b9622596aa2da6fe1fd42e88ffcf829"
server_key_version: "encryptionkey@0"
I20260808 21:00:41.538246 13991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:41.552345 13991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:41.554914 13991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:41.555106 13991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:41.560123 13991 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39439
I20260808 21:00:41.560263 14102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39439 every 8 connection(s)
I20260808 21:00:41.560624 13991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:41.561661 14103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:41.564383 14103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2: Bootstrap starting.
I20260808 21:00:41.565037 14103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:41.565030 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 13991
I20260808 21:00:41.565202 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:41.565397 16290 external_mini_cluster.cc:1428] Setting key 7c6ad1c4cf5a4eaac14c7444dd6932da
I20260808 21:00:41.565429 14103 log.cc:824] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:41.566696 14103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2: No bootstrap required, opened a new log
I20260808 21:00:41.569650 14103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ef1a53d45d4f2fb05f8eca873e53a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39439 } }
I20260808 21:00:41.569862 14103 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:41.569902 14103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59ef1a53d45d4f2fb05f8eca873e53a2, State: Initialized, Role: FOLLOWER
I20260808 21:00:41.570093 14103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ef1a53d45d4f2fb05f8eca873e53a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39439 } }
I20260808 21:00:41.570206 14103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:41.570259 14103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:41.570317 14103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:41.571080 14103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ef1a53d45d4f2fb05f8eca873e53a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39439 } }
I20260808 21:00:41.571206 14103 leader_election.cc:302] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59ef1a53d45d4f2fb05f8eca873e53a2; no voters:
I20260808 21:00:41.571414 14103 leader_election.cc:288] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:41.571501 14112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:41.571743 14112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [term 1 LEADER]: Becoming Leader. State: Replica: 59ef1a53d45d4f2fb05f8eca873e53a2, State: Running, Role: LEADER
I20260808 21:00:41.571755 14103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:41.571913 14112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ef1a53d45d4f2fb05f8eca873e53a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39439 } }
I20260808 21:00:41.572721 14115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59ef1a53d45d4f2fb05f8eca873e53a2. Latest consensus state: current_term: 1 leader_uuid: "59ef1a53d45d4f2fb05f8eca873e53a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ef1a53d45d4f2fb05f8eca873e53a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39439 } } }
I20260808 21:00:41.572885 14115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:41.572885 14113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59ef1a53d45d4f2fb05f8eca873e53a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ef1a53d45d4f2fb05f8eca873e53a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39439 } } }
I20260808 21:00:41.573027 14113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:41.573572 14123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:41.574153 14123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 21:00:41 dist-test-slave-c12x krb5kdc[13973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 21:00:41.576602 14123 catalog_manager.cc:1347] Generated new cluster ID: 5eca8dccee7c4806a71a76476ba0bf7c
I20260808 21:00:41.576665 14123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:41.579141 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:41.566149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:55337 (local address 127.0.0.1:39439)
0808 21:00:41.566339 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:41.566350 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:41.566388 (+ 38us) server_negotiation.cc:407] Connection header received
0808 21:00:41.567099 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:41.567110 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:41.567186 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:41.567360 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:41.568158 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:41.569885 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:41.570913 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:41.571665 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:41.572204 (+ 539us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:41.575309 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:41.575314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:41.575314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:41.575327 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:41.577287 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:41.577778 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:41.577782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:41.577782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:41.577889 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:41.578165 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:41.578166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:41.578167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:41.578451 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:41.578653 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:41.578752 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 21:00:41.578871 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260808 21:00:41.580724 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:41.591164 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:41.582257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:57979 (local address 127.0.0.1:39439)
0808 21:00:41.582300 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:41.582301 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:41.583765 (+ 1464us) server_negotiation.cc:407] Connection header received
0808 21:00:41.583842 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:41.583843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:41.583855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:41.584034 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:41.584704 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:41.585283 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:41.586116 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:41.586380 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:41.586474 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:41.588059 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:41.588060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:41.588061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:41.588069 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:41.589739 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:41.590181 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:41.590182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:41.590182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:41.590248 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:41.590510 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:41.590512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:41.590512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:41.590642 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:41.590776 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:41.590951 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 21:00:41.591027 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 21:00:41.595209 14123 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:41.595902 14123 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:41.604391 14123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59ef1a53d45d4f2fb05f8eca873e53a2: Generated new TSK 0
I20260808 21:00:41.604651 14123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:41.611114 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 13991
W20260808 21:00:41.628799 13970 connection.cc:535] client connection to 127.0.0.1:39439 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39439): Connection reset by peer (error 104)
2026-08-08T21:00:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1587 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:41 dist-test-slave-c12x krb5kdc[14146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:41 dist-test-slave-c12x krb5kdc[14146](info): set up 2 sockets
Aug 08 21:00:41 dist-test-slave-c12x krb5kdc[14146](info): commencing operation
krb5kdc: starting...
W20260808 21:00:42.862910 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:42 dist-test-slave-c12x krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:43.003122 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39985
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:43.091084 14162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:43.091382 14162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:43.091424 14162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:43.092667 14162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:43.092746 14162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:43.092775 14162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:43.092806 14162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:43.092837 14162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:43.094384 14162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:43.094681 14162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:43.094929 14162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:43.100118 14170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:43.100200 14173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:43.100173 14171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:43.100370 14162 server_base.cc:1034] running on GCE node
I20260808 21:00:43.100623 14162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:43.100893 14162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:43.102049 14162 hybrid_clock.cc:648] HybridClock initialized: now 1786222843102033 us; error 31 us; skew 500 ppm
Aug 08 21:00:43 dist-test-slave-c12x krb5kdc[14146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:43.104776 14162 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:43.105269 14162 webserver.cc:460] Webserver started at http://127.0.0.1:39871/ using document root <none> and password file <none>
I20260808 21:00:43.105635 14162 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:43.105690 14162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:43.105829 14162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:43.107077 14162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "3d4566d20d7b4ac2bedfa2b76826aac6"
format_stamp: "Formatted at 2026-08-08 21:00:43 on dist-test-slave-c12x"
server_key: "6eeb691341941f0c1de0b693bd50646c"
server_key_iv: "6d4b653f9e8a1349386d7e060325c3c9"
server_key_version: "encryptionkey@0"
I20260808 21:00:43.107542 14162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "3d4566d20d7b4ac2bedfa2b76826aac6"
format_stamp: "Formatted at 2026-08-08 21:00:43 on dist-test-slave-c12x"
server_key: "6eeb691341941f0c1de0b693bd50646c"
server_key_iv: "6d4b653f9e8a1349386d7e060325c3c9"
server_key_version: "encryptionkey@0"
I20260808 21:00:43.108976 14162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:43.109910 14186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:43.110209 14162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:43.110284 14162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "3d4566d20d7b4ac2bedfa2b76826aac6"
format_stamp: "Formatted at 2026-08-08 21:00:43 on dist-test-slave-c12x"
server_key: "6eeb691341941f0c1de0b693bd50646c"
server_key_iv: "6d4b653f9e8a1349386d7e060325c3c9"
server_key_version: "encryptionkey@0"
I20260808 21:00:43.110387 14162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:43.129731 14162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:43.132078 14162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:43.132243 14162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:43.136914 14162 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39985
I20260808 21:00:43.136976 14282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39985 every 8 connection(s)
I20260808 21:00:43.137349 14162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:43.138381 14283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:43.140256 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 14162
I20260808 21:00:43.140396 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:43.140632 16290 external_mini_cluster.cc:1428] Setting key 44c143396bbe352637ca9cb9977a4e46
I20260808 21:00:43.141952 14283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6: Bootstrap starting.
I20260808 21:00:43.142652 14283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:43.142961 14283 log.cc:824] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:43.143910 14283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6: No bootstrap required, opened a new log
I20260808 21:00:43.146579 14283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4566d20d7b4ac2bedfa2b76826aac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39985 } }
I20260808 21:00:43.146747 14283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:43.146773 14283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d4566d20d7b4ac2bedfa2b76826aac6, State: Initialized, Role: FOLLOWER
I20260808 21:00:43.146903 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:43.141238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:55427 (local address 127.0.0.1:39985)
0808 21:00:43.141436 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:43.141444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:43.141470 (+ 26us) server_negotiation.cc:407] Connection header received
0808 21:00:43.141880 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:43.141887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:43.141938 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:43.142114 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:43.142341 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:43.143621 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:43.144743 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:43.145172 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:43.145253 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:43.146107 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:43.146111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:43.146111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:43.146276 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:43.146309 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:43.146320 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:43.146387 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:43.146538 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 21:00:43.146627 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260808 21:00:43.146942 14283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4566d20d7b4ac2bedfa2b76826aac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39985 } }
I20260808 21:00:43.147073 14283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:43.147121 14283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:43.147167 14283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:43.147958 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:43.148130 14283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4566d20d7b4ac2bedfa2b76826aac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39985 } }
I20260808 21:00:43.148245 14283 leader_election.cc:302] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d4566d20d7b4ac2bedfa2b76826aac6; no voters:
I20260808 21:00:43.148459 14283 leader_election.cc:288] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:43.148525 14290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:43.148698 14290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [term 1 LEADER]: Becoming Leader. State: Replica: 3d4566d20d7b4ac2bedfa2b76826aac6, State: Running, Role: LEADER
I20260808 21:00:43.148742 14283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:43.149835 14290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4566d20d7b4ac2bedfa2b76826aac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39985 } }
I20260808 21:00:43.150609 14291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d4566d20d7b4ac2bedfa2b76826aac6. Latest consensus state: current_term: 1 leader_uuid: "3d4566d20d7b4ac2bedfa2b76826aac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4566d20d7b4ac2bedfa2b76826aac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39985 } } }
I20260808 21:00:43.150702 14291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:43.150880 14292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d4566d20d7b4ac2bedfa2b76826aac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d4566d20d7b4ac2bedfa2b76826aac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39985 } } }
I20260808 21:00:43.150929 14292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6 [sys.catalog]: This master's current role is: LEADER
W20260808 21:00:43.151508 14322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:00:43.151573 14322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 21:00:43.151665 14324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:43.151852 14324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:43.153393 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:43.149700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:54141 (local address 127.0.0.1:39985)
0808 21:00:43.149751 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:43.149753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:43.149838 (+ 85us) server_negotiation.cc:407] Connection header received
0808 21:00:43.149947 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:43.149949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:43.149961 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:43.150034 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:43.150239 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:43.151009 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:43.152051 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:43.152258 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:43.152316 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:43.153129 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:43.153130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:43.153130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:43.153136 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:43.153154 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:43.153158 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:43.153228 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:43.153304 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 21:00:43.153314 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 21:00:43.153687 14324 catalog_manager.cc:1347] Generated new cluster ID: b758ba1481a94bd89075c4c9ab2d6302
I20260808 21:00:43.153750 14324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:43.161703 14324 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:43.162367 14324 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:43.166523 14324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d4566d20d7b4ac2bedfa2b76826aac6: Generated new TSK 0
I20260808 21:00:43.166802 14324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:43.170606 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 14162
W20260808 21:00:43.186054 14143 connection.cc:535] client connection to 127.0.0.1:39985 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39985): Connection reset by peer (error 104)
2026-08-08T21:00:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:43 dist-test-slave-c12x krb5kdc[14330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:43 dist-test-slave-c12x krb5kdc[14330](info): set up 2 sockets
Aug 08 21:00:43 dist-test-slave-c12x krb5kdc[14330](info): commencing operation
krb5kdc: starting...
W20260808 21:00:44.417726 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:44 dist-test-slave-c12x krb5kdc[14330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:44.559502 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:44.647851 14351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:44.648164 14351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:44.648211 14351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:44.649504 14351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:44.649603 14351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:44.649629 14351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:44.649660 14351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:44.649691 14351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:44.651305 14351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:44.651639 14351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:44.651906 14351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:44.657382 14361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:44.657482 14357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:44.657380 14363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:44.657630 14351 server_base.cc:1034] running on GCE node
I20260808 21:00:44.658090 14351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:44.658470 14351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:44.659657 14351 hybrid_clock.cc:648] HybridClock initialized: now 1786222844659640 us; error 30 us; skew 500 ppm
Aug 08 21:00:44 dist-test-slave-c12x krb5kdc[14330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:44.662673 14351 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:44.663213 14351 webserver.cc:460] Webserver started at http://127.0.0.1:36663/ using document root <none> and password file <none>
I20260808 21:00:44.663704 14351 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:44.663802 14351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:44.663975 14351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:44.665114 14351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "ecbf91cdcf8046dca7bbacfc3b75b712"
format_stamp: "Formatted at 2026-08-08 21:00:44 on dist-test-slave-c12x"
server_key: "3a159989e7621107f233e9d9fe8d37dd"
server_key_iv: "bfa402ca20a92c51c479c066c107dc93"
server_key_version: "encryptionkey@0"
I20260808 21:00:44.665501 14351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "ecbf91cdcf8046dca7bbacfc3b75b712"
format_stamp: "Formatted at 2026-08-08 21:00:44 on dist-test-slave-c12x"
server_key: "3a159989e7621107f233e9d9fe8d37dd"
server_key_iv: "bfa402ca20a92c51c479c066c107dc93"
server_key_version: "encryptionkey@0"
I20260808 21:00:44.666965 14351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 21:00:44.667979 14370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:44.668255 14351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 21:00:44.668381 14351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "ecbf91cdcf8046dca7bbacfc3b75b712"
format_stamp: "Formatted at 2026-08-08 21:00:44 on dist-test-slave-c12x"
server_key: "3a159989e7621107f233e9d9fe8d37dd"
server_key_iv: "bfa402ca20a92c51c479c066c107dc93"
server_key_version: "encryptionkey@0"
I20260808 21:00:44.668448 14351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:44.672132 14351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:44.674736 14351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:44.674984 14351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:44.680166 14351 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40919
I20260808 21:00:44.680218 14453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40919 every 8 connection(s)
I20260808 21:00:44.680652 14351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:44.681933 14454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:44.684893 14454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712: Bootstrap starting.
I20260808 21:00:44.685694 14454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:44.686075 14454 log.cc:824] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:44.686096 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 14351
I20260808 21:00:44.686270 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:44.686556 16290 external_mini_cluster.cc:1428] Setting key 103fb3a3cd483b2dd819c3f3d4a71df7
I20260808 21:00:44.687161 14454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712: No bootstrap required, opened a new log
I20260808 21:00:44.689261 14454 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecbf91cdcf8046dca7bbacfc3b75b712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260808 21:00:44.689467 14454 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:44.689541 14454 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbf91cdcf8046dca7bbacfc3b75b712, State: Initialized, Role: FOLLOWER
I20260808 21:00:44.689738 14454 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecbf91cdcf8046dca7bbacfc3b75b712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260808 21:00:44.689859 14454 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:44.689952 14454 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:44.690019 14454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:44.691102 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:44.687270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:41633 (local address 127.0.0.1:40919)
0808 21:00:44.687537 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:44.687545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:44.687576 (+ 31us) server_negotiation.cc:407] Connection header received
0808 21:00:44.689258 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:44.689268 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:44.689318 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:44.689559 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:44.690564 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:44.690568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:44.690568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:44.690580 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:44.690629 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:44.690643 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:44.690740 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:44.690842 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 21:00:44.690927 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260808 21:00:44.691257 14454 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecbf91cdcf8046dca7bbacfc3b75b712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260808 21:00:44.691387 14454 leader_election.cc:302] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecbf91cdcf8046dca7bbacfc3b75b712; no voters:
I20260808 21:00:44.691682 14454 leader_election.cc:288] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:44.691761 14461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:44.691960 14461 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [term 1 LEADER]: Becoming Leader. State: Replica: ecbf91cdcf8046dca7bbacfc3b75b712, State: Running, Role: LEADER
I20260808 21:00:44.692056 14454 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:44.692067 14461 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecbf91cdcf8046dca7bbacfc3b75b712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } }
I20260808 21:00:44.692346 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:44.692672 14461 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecbf91cdcf8046dca7bbacfc3b75b712. Latest consensus state: current_term: 1 leader_uuid: "ecbf91cdcf8046dca7bbacfc3b75b712" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecbf91cdcf8046dca7bbacfc3b75b712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } } }
I20260808 21:00:44.692767 14461 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:44.692687 14464 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecbf91cdcf8046dca7bbacfc3b75b712" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecbf91cdcf8046dca7bbacfc3b75b712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40919 } } }
I20260808 21:00:44.692838 14464 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:44.693419 14476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:44.694123 14476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:44.696151 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:44.694324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:59191 (local address 127.0.0.1:40919)
0808 21:00:44.694369 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:44.694371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:44.694599 (+ 228us) server_negotiation.cc:407] Connection header received
0808 21:00:44.694629 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:44.694630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:44.694636 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:44.694707 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:44.695852 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:44.695853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:44.695854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:44.695858 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:44.695876 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:44.695881 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:44.695976 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:44.696048 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 21:00:44.696065 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 21:00:44.696260 14476 catalog_manager.cc:1347] Generated new cluster ID: 7f56f736aeda42ddb7e8770cfc9e3beb
I20260808 21:00:44.696305 14476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:44.706097 14476 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:44.706343 14476 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:44.710403 14476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecbf91cdcf8046dca7bbacfc3b75b712: Generated new TSK 0
I20260808 21:00:44.710665 14476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:44.719388 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 14351
2026-08-08T21:00:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:44 dist-test-slave-c12x krb5kdc[14498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:44 dist-test-slave-c12x krb5kdc[14498](info): set up 2 sockets
Aug 08 21:00:44 dist-test-slave-c12x krb5kdc[14498](info): commencing operation
krb5kdc: starting...
W20260808 21:00:46.055310 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:46 dist-test-slave-c12x krb5kdc[14498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:46.195614 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:46.284121 14517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:46.284390 14517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:46.284433 14517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:46.285614 14517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:46.285691 14517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:46.285718 14517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:46.285748 14517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:46.285768 14517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:46.287326 14517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41747
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:46.287639 14517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:46.287885 14517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:46.293404 14529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:46.293404 14524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:46.293431 14526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:46.293701 14517 server_base.cc:1034] running on GCE node
I20260808 21:00:46.293939 14517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:46.294291 14517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:46.295522 14517 hybrid_clock.cc:648] HybridClock initialized: now 1786222846295515 us; error 40 us; skew 500 ppm
Aug 08 21:00:46 dist-test-slave-c12x krb5kdc[14498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:46.304875 14517 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:46.305397 14517 webserver.cc:460] Webserver started at http://127.0.0.1:38443/ using document root <none> and password file <none>
I20260808 21:00:46.305684 14517 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:46.305765 14517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:46.305927 14517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:46.307065 14517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "b9bd517acdbf4ced966792afba90ba4a"
format_stamp: "Formatted at 2026-08-08 21:00:46 on dist-test-slave-c12x"
server_key: "d6b6737ae42d0b26b89b02edd55da473"
server_key_iv: "b2dcb1a66bd9d4072b7b8d80115c0cbf"
server_key_version: "encryptionkey@0"
I20260808 21:00:46.307541 14517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "b9bd517acdbf4ced966792afba90ba4a"
format_stamp: "Formatted at 2026-08-08 21:00:46 on dist-test-slave-c12x"
server_key: "d6b6737ae42d0b26b89b02edd55da473"
server_key_iv: "b2dcb1a66bd9d4072b7b8d80115c0cbf"
server_key_version: "encryptionkey@0"
I20260808 21:00:46.309008 14517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:46.309841 14541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:46.310079 14517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:46.310201 14517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "b9bd517acdbf4ced966792afba90ba4a"
format_stamp: "Formatted at 2026-08-08 21:00:46 on dist-test-slave-c12x"
server_key: "d6b6737ae42d0b26b89b02edd55da473"
server_key_iv: "b2dcb1a66bd9d4072b7b8d80115c0cbf"
server_key_version: "encryptionkey@0"
I20260808 21:00:46.310298 14517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:46.326730 14517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:46.329154 14517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:46.329352 14517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:46.333726 14517 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41747
I20260808 21:00:46.333762 14628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41747 every 8 connection(s)
I20260808 21:00:46.334160 14517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:46.335104 14631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:46.337528 14631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a: Bootstrap starting.
I20260808 21:00:46.338168 14631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:46.338482 14631 log.cc:824] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:46.339310 14631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a: No bootstrap required, opened a new log
I20260808 21:00:46.341003 14631 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9bd517acdbf4ced966792afba90ba4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41747 } }
I20260808 21:00:46.341156 14631 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:46.341178 14631 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9bd517acdbf4ced966792afba90ba4a, State: Initialized, Role: FOLLOWER
I20260808 21:00:46.341344 14631 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9bd517acdbf4ced966792afba90ba4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41747 } }
I20260808 21:00:46.341415 14631 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:46.341437 14631 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:46.341466 14631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:46.342057 14631 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9bd517acdbf4ced966792afba90ba4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41747 } }
I20260808 21:00:46.342147 14631 leader_election.cc:302] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9bd517acdbf4ced966792afba90ba4a; no voters:
I20260808 21:00:46.342383 14631 leader_election.cc:288] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:46.342444 14636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:46.342667 14631 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:46.342720 14636 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [term 1 LEADER]: Becoming Leader. State: Replica: b9bd517acdbf4ced966792afba90ba4a, State: Running, Role: LEADER
I20260808 21:00:46.342901 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 14517
I20260808 21:00:46.342917 14636 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9bd517acdbf4ced966792afba90ba4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41747 } }
I20260808 21:00:46.343065 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:46.343302 16290 external_mini_cluster.cc:1428] Setting key fc9c5950ce07210c92b128c7ff778e59
I20260808 21:00:46.343324 14639 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9bd517acdbf4ced966792afba90ba4a. Latest consensus state: current_term: 1 leader_uuid: "b9bd517acdbf4ced966792afba90ba4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9bd517acdbf4ced966792afba90ba4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41747 } } }
I20260808 21:00:46.343408 14639 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:46.343506 14638 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9bd517acdbf4ced966792afba90ba4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9bd517acdbf4ced966792afba90ba4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41747 } } }
I20260808 21:00:46.343619 14638 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:46.344045 14656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:46.344656 14656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:46.347994 14656 catalog_manager.cc:1347] Generated new cluster ID: 31d5e6e02cbb4511ae89ec3bcf85f1b5
I20260808 21:00:46.348061 14656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 21:00:46 dist-test-slave-c12x krb5kdc[14498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 21:00:46.355141 14657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:46.343911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:42601 (local address 127.0.0.1:41747)
0808 21:00:46.344111 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:46.344120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:46.344726 (+ 606us) server_negotiation.cc:407] Connection header received
0808 21:00:46.344862 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:46.344870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:46.344935 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:46.345071 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:46.345986 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:46.347373 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:46.348476 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:46.348906 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:46.349052 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:46.351690 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:46.351695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:46.351695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:46.351709 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:46.353469 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:46.353905 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:46.353908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:46.353909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:46.354008 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:46.354253 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:46.354255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:46.354255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:46.354506 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:46.354697 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:46.354785 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 21:00:46.354906 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":118,"threads_started":1}
I20260808 21:00:46.356137 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:46.367277 14657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:46.357542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:58441 (local address 127.0.0.1:41747)
0808 21:00:46.357584 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:46.357586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:46.359645 (+ 2059us) server_negotiation.cc:407] Connection header received
0808 21:00:46.359736 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:46.359737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:46.359749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:46.359806 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:46.360365 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:46.360986 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:46.362002 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:46.362254 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:46.362364 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:46.363996 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:46.363998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:46.363998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:46.364007 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:46.365716 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:46.366210 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:46.366212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:46.366212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:46.366283 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:46.366584 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:46.366585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:46.366585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:46.366722 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:46.366872 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:46.367058 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 21:00:46.367134 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 21:00:46.371549 14656 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:46.372149 14656 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:46.379884 14656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9bd517acdbf4ced966792afba90ba4a: Generated new TSK 0
I20260808 21:00:46.380125 14656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:46.389468 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 14517
W20260808 21:00:46.404624 14496 connection.cc:535] client connection to 127.0.0.1:41747 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41747): Connection reset by peer (error 104)
2026-08-08T21:00:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1670 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:46 dist-test-slave-c12x krb5kdc[14679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:46 dist-test-slave-c12x krb5kdc[14679](info): set up 2 sockets
Aug 08 21:00:46 dist-test-slave-c12x krb5kdc[14679](info): commencing operation
krb5kdc: starting...
W20260808 21:00:47.636852 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:47 dist-test-slave-c12x krb5kdc[14679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:47.780200 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:47.870153 14695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:47.870405 14695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:47.870431 14695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:47.871801 14695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:47.871856 14695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:47.871871 14695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:47.871884 14695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:47.871896 14695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:47.873548 14695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33911
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:47.873848 14695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:47.874121 14695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:47.879568 14701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:47.879585 14704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:47.879842 14702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:47.879959 14695 server_base.cc:1034] running on GCE node
I20260808 21:00:47.880242 14695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:47.880636 14695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:47.881800 14695 hybrid_clock.cc:648] HybridClock initialized: now 1786222847881778 us; error 39 us; skew 500 ppm
Aug 08 21:00:47 dist-test-slave-c12x krb5kdc[14679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:47.889969 14695 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:47.890604 14695 webserver.cc:460] Webserver started at http://127.0.0.1:46861/ using document root <none> and password file <none>
I20260808 21:00:47.890904 14695 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:47.890986 14695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:47.891156 14695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:47.892447 14695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "3dbf101caac642268b65afd39b849c1c"
format_stamp: "Formatted at 2026-08-08 21:00:47 on dist-test-slave-c12x"
server_key: "a7538cf38af5d9759aecc29b4b1ba5b1"
server_key_iv: "59f8b57dc56c2580a3f4b9305ee5a46d"
server_key_version: "encryptionkey@0"
I20260808 21:00:47.892858 14695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "3dbf101caac642268b65afd39b849c1c"
format_stamp: "Formatted at 2026-08-08 21:00:47 on dist-test-slave-c12x"
server_key: "a7538cf38af5d9759aecc29b4b1ba5b1"
server_key_iv: "59f8b57dc56c2580a3f4b9305ee5a46d"
server_key_version: "encryptionkey@0"
I20260808 21:00:47.894503 14695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 21:00:47.895601 14719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:47.895888 14695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:47.896024 14695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "3dbf101caac642268b65afd39b849c1c"
format_stamp: "Formatted at 2026-08-08 21:00:47 on dist-test-slave-c12x"
server_key: "a7538cf38af5d9759aecc29b4b1ba5b1"
server_key_iv: "59f8b57dc56c2580a3f4b9305ee5a46d"
server_key_version: "encryptionkey@0"
I20260808 21:00:47.896124 14695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:47.911309 14695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:47.913985 14695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:47.914191 14695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:47.919072 14695 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33911
I20260808 21:00:47.919091 14793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33911 every 8 connection(s)
I20260808 21:00:47.919576 14695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:47.920583 14795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:47.923013 14795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c: Bootstrap starting.
I20260808 21:00:47.923738 14795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:47.924091 14795 log.cc:824] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:47.924932 14795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c: No bootstrap required, opened a new log
I20260808 21:00:47.926617 14795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dbf101caac642268b65afd39b849c1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33911 } }
I20260808 21:00:47.926771 14795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:47.926795 14795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dbf101caac642268b65afd39b849c1c, State: Initialized, Role: FOLLOWER
I20260808 21:00:47.926972 14795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dbf101caac642268b65afd39b849c1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33911 } }
I20260808 21:00:47.927052 14795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:47.927075 14795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:47.927104 14795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:47.927385 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 14695
I20260808 21:00:47.927564 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:47.927892 14795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dbf101caac642268b65afd39b849c1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33911 } }
I20260808 21:00:47.928021 16290 external_mini_cluster.cc:1428] Setting key 8d79a6d9a0dff35fb0c6e8b161318f9b
I20260808 21:00:47.927991 14795 leader_election.cc:302] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3dbf101caac642268b65afd39b849c1c; no voters:
I20260808 21:00:47.928218 14795 leader_election.cc:288] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:47.928267 14799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:47.928416 14799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [term 1 LEADER]: Becoming Leader. State: Replica: 3dbf101caac642268b65afd39b849c1c, State: Running, Role: LEADER
I20260808 21:00:47.928539 14795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:47.928574 14799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dbf101caac642268b65afd39b849c1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33911 } }
I20260808 21:00:47.929011 14804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3dbf101caac642268b65afd39b849c1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dbf101caac642268b65afd39b849c1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33911 } } }
I20260808 21:00:47.929139 14802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3dbf101caac642268b65afd39b849c1c. Latest consensus state: current_term: 1 leader_uuid: "3dbf101caac642268b65afd39b849c1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dbf101caac642268b65afd39b849c1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33911 } } }
I20260808 21:00:47.929206 14802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:47.929188 14804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:47.929663 14814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:47.930596 14814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:47.932323 14814 catalog_manager.cc:1347] Generated new cluster ID: 7d6c5da3fc06481a8bf21409feef1629
I20260808 21:00:47.932384 14814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:47.935827 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:47.928771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:47269 (local address 127.0.0.1:33911)
0808 21:00:47.929534 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:47.929557 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:47.930097 (+ 540us) server_negotiation.cc:407] Connection header received
0808 21:00:47.930272 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:47.930280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:47.930346 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:47.930506 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:47.930940 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:47.932176 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:47.933341 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:47.933941 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:47.934795 (+ 854us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:47.934950 (+ 155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:47.934953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:47.934953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:47.935172 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:47.935205 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:47.935219 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:47.935528 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:47.935573 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 21:00:47.935602 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":119,"threads_started":1}
I20260808 21:00:47.937009 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:47.950263 14814 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:47.950846 14814 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:47.953639 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:47.949632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:40519 (local address 127.0.0.1:33911)
0808 21:00:47.949682 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:47.949684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:47.949872 (+ 188us) server_negotiation.cc:407] Connection header received
0808 21:00:47.949947 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:47.949948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:47.949972 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:47.950055 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:47.950235 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:47.951021 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:47.952102 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:47.952441 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:47.952519 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:47.953323 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:47.953324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:47.953325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:47.953331 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:47.953348 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:47.953353 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:47.953421 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:47.953529 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 21:00:47.953552 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 21:00:47.955802 14814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3dbf101caac642268b65afd39b849c1c: Generated new TSK 0
I20260808 21:00:47.956023 14814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:47.961278 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 14695
W20260808 21:00:47.973562 14676 connection.cc:535] client connection to 127.0.0.1:33911 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33911): Connection reset by peer (error 104)
2026-08-08T21:00:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1568 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:48 dist-test-slave-c12x krb5kdc[14846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:48 dist-test-slave-c12x krb5kdc[14846](info): set up 2 sockets
Aug 08 21:00:48 dist-test-slave-c12x krb5kdc[14846](info): commencing operation
krb5kdc: starting...
W20260808 21:00:49.225708 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:49 dist-test-slave-c12x krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 21:00:49.363883 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:49.449062 14874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:49.449321 14874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:49.449348 14874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:49.450678 14874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:49.450736 14874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:49.450752 14874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:49.450764 14874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:49.450778 14874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:49.452662 14874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:49.452910 14874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:49.453109 14874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:49.458640 14880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:49.458712 14882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:49.458640 14886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:49.458958 14874 server_base.cc:1034] running on GCE node
I20260808 21:00:49.459333 14874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:49.459712 14874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:49.460964 14874 hybrid_clock.cc:648] HybridClock initialized: now 1786222849460948 us; error 33 us; skew 500 ppm
Aug 08 21:00:49 dist-test-slave-c12x krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:49.464027 14874 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 21:00:49.464550 14874 webserver.cc:460] Webserver started at http://127.0.0.1:42415/ using document root <none> and password file <none>
I20260808 21:00:49.464812 14874 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:49.464886 14874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:49.465050 14874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:49.466161 14874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "e19c619747a54925ae4926fb713983ea"
format_stamp: "Formatted at 2026-08-08 21:00:49 on dist-test-slave-c12x"
server_key: "cf73e41b8d4a7ec8384513ebe70824e8"
server_key_iv: "81359ff0f6d7d086408c1097c8708654"
server_key_version: "encryptionkey@0"
I20260808 21:00:49.466547 14874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "e19c619747a54925ae4926fb713983ea"
format_stamp: "Formatted at 2026-08-08 21:00:49 on dist-test-slave-c12x"
server_key: "cf73e41b8d4a7ec8384513ebe70824e8"
server_key_iv: "81359ff0f6d7d086408c1097c8708654"
server_key_version: "encryptionkey@0"
I20260808 21:00:49.468153 14874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 21:00:49.469159 14895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:49.469470 14874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:00:49.469650 14874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "e19c619747a54925ae4926fb713983ea"
format_stamp: "Formatted at 2026-08-08 21:00:49 on dist-test-slave-c12x"
server_key: "cf73e41b8d4a7ec8384513ebe70824e8"
server_key_iv: "81359ff0f6d7d086408c1097c8708654"
server_key_version: "encryptionkey@0"
I20260808 21:00:49.469755 14874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:49.474140 14874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:49.476768 14874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:49.476963 14874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:49.482244 14874 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39319
I20260808 21:00:49.482296 14974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39319 every 8 connection(s)
I20260808 21:00:49.482681 14874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:49.484009 14975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:49.486749 14975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea: Bootstrap starting.
I20260808 21:00:49.487346 14975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:49.487715 14975 log.cc:824] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:49.488528 14975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea: No bootstrap required, opened a new log
I20260808 21:00:49.490337 14975 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19c619747a54925ae4926fb713983ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39319 } }
I20260808 21:00:49.490518 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 14874
I20260808 21:00:49.490556 14975 raft_consensus.cc:374] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:49.490602 14975 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19c619747a54925ae4926fb713983ea, State: Initialized, Role: FOLLOWER
I20260808 21:00:49.490654 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:49.490860 16290 external_mini_cluster.cc:1428] Setting key e559ce31a76054e2126f39c1cd220ec2
I20260808 21:00:49.490813 14975 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19c619747a54925ae4926fb713983ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39319 } }
I20260808 21:00:49.490907 14975 raft_consensus.cc:388] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:49.490947 14975 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:49.490993 14975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:49.492270 14975 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19c619747a54925ae4926fb713983ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39319 } }
I20260808 21:00:49.492403 14975 leader_election.cc:302] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e19c619747a54925ae4926fb713983ea; no voters:
I20260808 21:00:49.492588 14975 leader_election.cc:288] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:49.492663 14987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:49.492872 14987 raft_consensus.cc:686] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [term 1 LEADER]: Becoming Leader. State: Replica: e19c619747a54925ae4926fb713983ea, State: Running, Role: LEADER
I20260808 21:00:49.492898 14975 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:49.493057 14987 consensus_queue.cc:237] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19c619747a54925ae4926fb713983ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39319 } }
I20260808 21:00:49.493489 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:49.491575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:36759 (local address 127.0.0.1:39319)
0808 21:00:49.491865 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:49.491874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:49.491917 (+ 43us) server_negotiation.cc:407] Connection header received
0808 21:00:49.492021 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:49.492027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:49.492051 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:49.492171 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:49.492965 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:49.492970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:49.492970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:49.492975 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:49.493026 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:49.493041 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:49.493114 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:49.493238 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 21:00:49.493287 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":181,"threads_started":1}
I20260808 21:00:49.493781 14989 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader e19c619747a54925ae4926fb713983ea. Latest consensus state: current_term: 1 leader_uuid: "e19c619747a54925ae4926fb713983ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19c619747a54925ae4926fb713983ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39319 } } }
I20260808 21:00:49.493880 14989 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:49.494264 14999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:49.494432 14988 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e19c619747a54925ae4926fb713983ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19c619747a54925ae4926fb713983ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39319 } } }
I20260808 21:00:49.494702 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:49.494508 14988 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:49.495682 14999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:49.497757 14999 catalog_manager.cc:1347] Generated new cluster ID: f0e9698c94334573a35d59a7e48b0e7e
I20260808 21:00:49.497851 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:49.496418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:47345 (local address 127.0.0.1:39319)
0808 21:00:49.496460 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:49.496462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:49.496601 (+ 139us) server_negotiation.cc:407] Connection header received
0808 21:00:49.496675 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:49.496676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:49.496682 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:49.496732 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:49.497490 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:49.497491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:49.497492 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:49.497496 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:49.497514 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:49.497519 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:49.497596 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:49.497699 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 21:00:49.497716 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 21:00:49.497908 14999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:49.507305 14999 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:49.507608 14999 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:49.511648 14999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e19c619747a54925ae4926fb713983ea: Generated new TSK 0
I20260808 21:00:49.511852 14999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:49.517321 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 14874
2026-08-08T21:00:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:49 dist-test-slave-c12x krb5kdc[15018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:49 dist-test-slave-c12x krb5kdc[15018](info): set up 2 sockets
Aug 08 21:00:49 dist-test-slave-c12x krb5kdc[15018](info): commencing operation
krb5kdc: starting...
W20260808 21:00:50.784066 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:50 dist-test-slave-c12x krb5kdc[15018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:00:50.933341 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:39721
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:35129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:39721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:51.027038 15034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:51.027272 15034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:51.027298 15034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:51.028817 15034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:51.028883 15034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:51.028899 15034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:51.028913 15034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:51.028929 15034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:51.030771 15034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:35129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:39721
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:39721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:51.031010 15034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:51.031201 15034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:51.036762 15050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:51.036831 15044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:51.036762 15045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:51.037099 15034 server_base.cc:1034] running on GCE node
I20260808 21:00:51.037367 15034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:51.037727 15034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:51.038934 15034 hybrid_clock.cc:648] HybridClock initialized: now 1786222851038910 us; error 48 us; skew 500 ppm
Aug 08 21:00:51 dist-test-slave-c12x krb5kdc[15018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222851, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:51.041940 15034 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:51.042577 15034 webserver.cc:460] Webserver started at http://127.15.232.190:43811/ using document root <none> and password file <none>
I20260808 21:00:51.042855 15034 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:51.042943 15034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:51.043135 15034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:51.044474 15034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "eafd420a6cba4b258637b7622970c479"
format_stamp: "Formatted at 2026-08-08 21:00:51 on dist-test-slave-c12x"
server_key: "857de87aa185daae75feec273db336ae"
server_key_iv: "27b5d6f7c3cc215afbf722006168c568"
server_key_version: "encryptionkey@0"
I20260808 21:00:51.044909 15034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "eafd420a6cba4b258637b7622970c479"
format_stamp: "Formatted at 2026-08-08 21:00:51 on dist-test-slave-c12x"
server_key: "857de87aa185daae75feec273db336ae"
server_key_iv: "27b5d6f7c3cc215afbf722006168c568"
server_key_version: "encryptionkey@0"
I20260808 21:00:51.046474 15034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 21:00:51.047359 15062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:51.047600 15034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:51.047720 15034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "eafd420a6cba4b258637b7622970c479"
format_stamp: "Formatted at 2026-08-08 21:00:51 on dist-test-slave-c12x"
server_key: "857de87aa185daae75feec273db336ae"
server_key_iv: "27b5d6f7c3cc215afbf722006168c568"
server_key_version: "encryptionkey@0"
I20260808 21:00:51.047782 15034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:51.073130 15034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:51.075716 15034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:51.075836 15034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:51.079849 15034 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:39721
I20260808 21:00:51.079970 15144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:39721 every 8 connection(s)
I20260808 21:00:51.080297 15034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:51.081138 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 15034
I20260808 21:00:51.081314 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:51.081445 15145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:51.081636 16290 external_mini_cluster.cc:1428] Setting key af57c2508baff0845fd4c60d17991c84
I20260808 21:00:51.084897 15145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479: Bootstrap starting.
I20260808 21:00:51.085724 15145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:51.086045 15145 log.cc:824] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:51.086875 15145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479: No bootstrap required, opened a new log
Aug 08 21:00:51 dist-test-slave-c12x krb5kdc[15018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:51.089561 15145 raft_consensus.cc:348] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafd420a6cba4b258637b7622970c479" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39721 } }
I20260808 21:00:51.089715 15145 raft_consensus.cc:374] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:51.089736 15145 raft_consensus.cc:729] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eafd420a6cba4b258637b7622970c479, State: Initialized, Role: FOLLOWER
I20260808 21:00:51.089993 15145 consensus_queue.cc:260] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafd420a6cba4b258637b7622970c479" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39721 } }
I20260808 21:00:51.090101 15145 raft_consensus.cc:388] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:51.090149 15145 raft_consensus.cc:482] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:51.090200 15145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:51.091060 15145 raft_consensus.cc:504] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafd420a6cba4b258637b7622970c479" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39721 } }
I20260808 21:00:51.091197 15145 leader_election.cc:302] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eafd420a6cba4b258637b7622970c479; no voters:
I20260808 21:00:51.091387 15145 leader_election.cc:288] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:51.091549 15151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:51.091681 15145 sys_catalog.cc:564] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:51.091765 15151 raft_consensus.cc:686] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [term 1 LEADER]: Becoming Leader. State: Replica: eafd420a6cba4b258637b7622970c479, State: Running, Role: LEADER
I20260808 21:00:51.091879 15151 consensus_queue.cc:237] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafd420a6cba4b258637b7622970c479" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39721 } }
I20260808 21:00:51.092296 15151 sys_catalog.cc:455] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eafd420a6cba4b258637b7622970c479. Latest consensus state: current_term: 1 leader_uuid: "eafd420a6cba4b258637b7622970c479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafd420a6cba4b258637b7622970c479" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39721 } } }
I20260808 21:00:51.092303 15154 sys_catalog.cc:455] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eafd420a6cba4b258637b7622970c479" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eafd420a6cba4b258637b7622970c479" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39721 } } }
I20260808 21:00:51.092504 15154 sys_catalog.cc:458] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:51.092757 15151 sys_catalog.cc:458] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479 [sys.catalog]: This master's current role is: LEADER
W20260808 21:00:51.093701 15170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:00:51.093775 15170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 21:00:51.093865 15162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:51.094046 15162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:51.094286 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:51.082260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36937 (local address 127.15.232.190:39721)
0808 21:00:51.082489 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:51.082497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:51.082525 (+ 28us) server_negotiation.cc:407] Connection header received
0808 21:00:51.083016 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:51.083023 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:51.083072 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:51.083196 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:51.084099 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:51.085076 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:51.086223 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:51.086693 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:51.086800 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:51.089676 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:51.089680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:51.089681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:51.089698 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:51.091969 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:51.092617 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:51.092621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:51.092622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:51.092746 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:51.093104 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:51.093105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:51.093105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:51.093486 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:51.093665 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:51.093968 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 21:00:51.094095 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":120,"threads_started":1}
I20260808 21:00:51.095559 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:51.097424 15162 catalog_manager.cc:1347] Generated new cluster ID: cc8de900935c41af8321866ea6f2e2b3
I20260808 21:00:51.097486 15162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:51.105623 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:51.097257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38877 (local address 127.15.232.190:39721)
0808 21:00:51.097297 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:51.097299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:51.097411 (+ 112us) server_negotiation.cc:407] Connection header received
0808 21:00:51.097500 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:51.097501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:51.097513 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:51.097586 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:51.098251 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:51.099145 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:51.100227 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:51.100628 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:51.100718 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:51.102260 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:51.102262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:51.102262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:51.102271 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:51.104180 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:51.104653 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:51.104654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:51.104655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:51.104729 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:51.104993 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:51.104994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:51.104994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:51.105156 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:51.105282 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:51.105409 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 21:00:51.105487 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 21:00:51.112589 15162 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:51.113262 15162 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:51.118449 15162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eafd420a6cba4b258637b7622970c479: Generated new TSK 0
I20260808 21:00:51.118691 15162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:51.123994 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 15034
W20260808 21:00:51.136673 15015 connection.cc:535] client connection to 127.15.232.190:39721 recv error: Network error: failed to read from TLS socket (remote: 127.15.232.190:39721): Connection reset by peer (error 104)
2026-08-08T21:00:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1604 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:51 dist-test-slave-c12x krb5kdc[15184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:51 dist-test-slave-c12x krb5kdc[15184](info): set up 2 sockets
Aug 08 21:00:51 dist-test-slave-c12x krb5kdc[15184](info): commencing operation
krb5kdc: starting...
W20260808 21:00:52.374061 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:52 dist-test-slave-c12x krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:00:52.507477 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33491
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:39969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:33491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:52.593472 15201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:52.593755 15201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:52.593797 15201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:52.594980 15201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:52.595058 15201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:52.595086 15201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:52.595114 15201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:52.595142 15201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:52.596762 15201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:39969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:33491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:33491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:52.597072 15201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:52.597311 15201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:52.602332 15207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:52.602392 15210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:52.602432 15208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:52.602831 15201 server_base.cc:1034] running on GCE node
I20260808 21:00:52.603129 15201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:52.603396 15201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:52.604602 15201 hybrid_clock.cc:648] HybridClock initialized: now 1786222852604584 us; error 33 us; skew 500 ppm
Aug 08 21:00:52 dist-test-slave-c12x krb5kdc[15184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222852, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:52.611279 15201 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:52.611843 15201 webserver.cc:460] Webserver started at http://127.15.232.190:41709/ using document root <none> and password file <none>
I20260808 21:00:52.612106 15201 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:52.612175 15201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:52.612321 15201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:52.613492 15201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "5ec93026471848ea98c747ad0925194e"
format_stamp: "Formatted at 2026-08-08 21:00:52 on dist-test-slave-c12x"
server_key: "cc7fa2862c9f0801876a168605b1a801"
server_key_iv: "bd3bed8f0a8f13a66df224f270fedd46"
server_key_version: "encryptionkey@0"
I20260808 21:00:52.613864 15201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "5ec93026471848ea98c747ad0925194e"
format_stamp: "Formatted at 2026-08-08 21:00:52 on dist-test-slave-c12x"
server_key: "cc7fa2862c9f0801876a168605b1a801"
server_key_iv: "bd3bed8f0a8f13a66df224f270fedd46"
server_key_version: "encryptionkey@0"
I20260808 21:00:52.615530 15201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260808 21:00:52.616330 15221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:52.616523 15201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:52.616639 15201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "5ec93026471848ea98c747ad0925194e"
format_stamp: "Formatted at 2026-08-08 21:00:52 on dist-test-slave-c12x"
server_key: "cc7fa2862c9f0801876a168605b1a801"
server_key_iv: "bd3bed8f0a8f13a66df224f270fedd46"
server_key_version: "encryptionkey@0"
I20260808 21:00:52.616719 15201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:52.633106 15201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:52.635576 15201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:52.635787 15201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:52.640674 15201 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:33491
I20260808 21:00:52.640729 15302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:33491 every 8 connection(s)
I20260808 21:00:52.641161 15201 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:52.642156 15304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:52.643702 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 15201
I20260808 21:00:52.643852 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:52.644073 16290 external_mini_cluster.cc:1428] Setting key e65588ac06b5222bad403cac2f9b822b
I20260808 21:00:52.645062 15304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e: Bootstrap starting.
I20260808 21:00:52.645819 15304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:52.646189 15304 log.cc:824] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:52.647019 15304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e: No bootstrap required, opened a new log
I20260808 21:00:52.649333 15304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec93026471848ea98c747ad0925194e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33491 } }
I20260808 21:00:52.649533 15304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:52.649577 15304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec93026471848ea98c747ad0925194e, State: Initialized, Role: FOLLOWER
I20260808 21:00:52.649736 15304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec93026471848ea98c747ad0925194e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33491 } }
I20260808 21:00:52.649832 15304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:52.649866 15304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:52.649919 15304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:52.650588 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:52.644681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60079 (local address 127.15.232.190:33491)
0808 21:00:52.644906 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:52.644916 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:52.644950 (+ 34us) server_negotiation.cc:407] Connection header received
0808 21:00:52.645576 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:52.645586 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:52.645648 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:52.645808 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:52.646063 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:52.647359 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:52.648268 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:52.648985 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:52.649115 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:52.649863 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:52.649867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:52.649867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:52.650088 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:52.650128 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:52.650143 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:52.650234 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:52.650322 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 21:00:52.650351 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":101,"threads_started":1}
I20260808 21:00:52.650702 15304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec93026471848ea98c747ad0925194e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33491 } }
I20260808 21:00:52.650978 15304 leader_election.cc:302] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ec93026471848ea98c747ad0925194e; no voters:
I20260808 21:00:52.651222 15304 leader_election.cc:288] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:52.651252 15314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:52.651374 15314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [term 1 LEADER]: Becoming Leader. State: Replica: 5ec93026471848ea98c747ad0925194e, State: Running, Role: LEADER
I20260808 21:00:52.651580 15304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:52.651932 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:52.651549 15314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec93026471848ea98c747ad0925194e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33491 } }
I20260808 21:00:52.653083 15314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ec93026471848ea98c747ad0925194e. Latest consensus state: current_term: 1 leader_uuid: "5ec93026471848ea98c747ad0925194e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec93026471848ea98c747ad0925194e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33491 } } }
I20260808 21:00:52.653235 15314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:52.653282 15315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ec93026471848ea98c747ad0925194e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ec93026471848ea98c747ad0925194e" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 33491 } } }
I20260808 21:00:52.653360 15315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:52.653587 15340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:52.654263 15340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:52.655892 15340 catalog_manager.cc:1347] Generated new cluster ID: f116bebdf76145898022f8d3102ff1c9
I20260808 21:00:52.655958 15340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:52.657351 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:52.653706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42475 (local address 127.15.232.190:33491)
0808 21:00:52.653748 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:52.653751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:52.654039 (+ 288us) server_negotiation.cc:407] Connection header received
0808 21:00:52.654113 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:52.654114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:52.654127 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:52.654183 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:52.654400 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:52.655249 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:52.656086 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:52.656292 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:52.656364 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:52.657085 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:52.657087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:52.657087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:52.657092 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:52.657107 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:52.657112 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:52.657181 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:52.657248 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 21:00:52.657258 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 21:00:52.680657 15340 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:52.681420 15340 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:52.694027 15340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ec93026471848ea98c747ad0925194e: Generated new TSK 0
I20260808 21:00:52.694391 15340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:52.698252 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 15201
W20260808 21:00:52.712685 15182 connection.cc:535] client connection to 127.15.232.190:33491 recv error: Network error: failed to read from TLS socket (remote: 127.15.232.190:33491): Connection reset by peer (error 104)
2026-08-08T21:00:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:52 dist-test-slave-c12x krb5kdc[15352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:52 dist-test-slave-c12x krb5kdc[15352](info): set up 2 sockets
Aug 08 21:00:52 dist-test-slave-c12x krb5kdc[15352](info): commencing operation
krb5kdc: starting...
W20260808 21:00:53.990751 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:54 dist-test-slave-c12x krb5kdc[15352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:00:54.134754 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37317
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:37317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:54.220281 15373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:54.220539 15373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:54.220580 15373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:54.221792 15373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:54.221864 15373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:54.221890 15373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:54.221921 15373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:54.221951 15373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:54.223595 15373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:37317
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:54.223886 15373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:54.224121 15373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:54.229568 15378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:54.229599 15379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:54.229588 15373 server_base.cc:1034] running on GCE node
W20260808 21:00:54.229761 15381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:54.230074 15373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:54.230458 15373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:54.231622 15373 hybrid_clock.cc:648] HybridClock initialized: now 1786222854231597 us; error 42 us; skew 500 ppm
Aug 08 21:00:54 dist-test-slave-c12x krb5kdc[15352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:54.234659 15373 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:54.235174 15373 webserver.cc:460] Webserver started at http://127.15.232.190:44501/ using document root <none> and password file <none>
I20260808 21:00:54.235473 15373 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:54.235549 15373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:54.235735 15373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:54.237135 15373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f"
format_stamp: "Formatted at 2026-08-08 21:00:54 on dist-test-slave-c12x"
server_key: "16c964e16f5a07569f95bf1bbd7c0a64"
server_key_iv: "4643291a3e2b6b79dbfb4700d30da743"
server_key_version: "encryptionkey@0"
I20260808 21:00:54.237480 15373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f"
format_stamp: "Formatted at 2026-08-08 21:00:54 on dist-test-slave-c12x"
server_key: "16c964e16f5a07569f95bf1bbd7c0a64"
server_key_iv: "4643291a3e2b6b79dbfb4700d30da743"
server_key_version: "encryptionkey@0"
I20260808 21:00:54.239018 15373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 21:00:54.239951 15390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:54.240170 15373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 21:00:54.240259 15373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f"
format_stamp: "Formatted at 2026-08-08 21:00:54 on dist-test-slave-c12x"
server_key: "16c964e16f5a07569f95bf1bbd7c0a64"
server_key_iv: "4643291a3e2b6b79dbfb4700d30da743"
server_key_version: "encryptionkey@0"
I20260808 21:00:54.240329 15373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:54.244360 15373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:54.246657 15373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:54.246788 15373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:54.251590 15373 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:37317
I20260808 21:00:54.251628 15466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:37317 every 8 connection(s)
I20260808 21:00:54.252012 15373 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:54.253129 15468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:54.255713 15468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f: Bootstrap starting.
I20260808 21:00:54.256351 15468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:54.256659 15468 log.cc:824] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:54.257506 15468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f: No bootstrap required, opened a new log
I20260808 21:00:54.259182 15468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37317 } }
I20260808 21:00:54.259347 15468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:54.259397 15468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85fe1ba3507f4f37ae1c38bc7ef8be6f, State: Initialized, Role: FOLLOWER
I20260808 21:00:54.259636 15468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37317 } }
I20260808 21:00:54.259740 15468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:54.259786 15468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:54.259838 15468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:54.261047 15468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37317 } }
I20260808 21:00:54.261126 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 15373
I20260808 21:00:54.261204 15468 leader_election.cc:302] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85fe1ba3507f4f37ae1c38bc7ef8be6f; no voters:
I20260808 21:00:54.261265 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:54.261461 16290 external_mini_cluster.cc:1428] Setting key 3ce34ecb45702d7cb5bf95319756204e
I20260808 21:00:54.261463 15468 leader_election.cc:288] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:54.261488 15475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:54.261617 15475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [term 1 LEADER]: Becoming Leader. State: Replica: 85fe1ba3507f4f37ae1c38bc7ef8be6f, State: Running, Role: LEADER
I20260808 21:00:54.261765 15468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:54.261770 15475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37317 } }
I20260808 21:00:54.262259 15476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85fe1ba3507f4f37ae1c38bc7ef8be6f. Latest consensus state: current_term: 1 leader_uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37317 } } }
I20260808 21:00:54.262300 15478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85fe1ba3507f4f37ae1c38bc7ef8be6f" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37317 } } }
I20260808 21:00:54.262451 15478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:54.262698 15476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:54.263077 15491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:54.264181 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:54.262124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55255 (local address 127.15.232.190:37317)
0808 21:00:54.262478 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:54.262496 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:54.262530 (+ 34us) server_negotiation.cc:407] Connection header received
0808 21:00:54.262656 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:54.262664 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:54.262697 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:54.263144 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:54.263737 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:54.263741 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:54.263741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:54.263745 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:54.263783 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:54.263797 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:54.263859 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:54.263937 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 21:00:54.263981 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":77,"threads_started":1}
I20260808 21:00:54.264430 15491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:54.265254 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:54.266299 15491 catalog_manager.cc:1347] Generated new cluster ID: f3392009cf824ca89d740acd710be5f8
I20260808 21:00:54.266353 15491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:54.268751 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:54.267205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38111 (local address 127.15.232.190:37317)
0808 21:00:54.267254 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:54.267256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:54.267645 (+ 389us) server_negotiation.cc:407] Connection header received
0808 21:00:54.267728 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:54.267729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:54.267735 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:54.267789 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:54.268518 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:54.268519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:54.268519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:54.268523 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:54.268538 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:54.268543 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:54.268589 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:54.268664 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 21:00:54.268678 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 21:00:54.305923 15491 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:54.306133 15491 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:54.317185 15491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85fe1ba3507f4f37ae1c38bc7ef8be6f: Generated new TSK 0
I20260808 21:00:54.317408 15491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:54.328343 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 15373
2026-08-08T21:00:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1627 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:54 dist-test-slave-c12x krb5kdc[15522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:54 dist-test-slave-c12x krb5kdc[15522](info): set up 2 sockets
Aug 08 21:00:54 dist-test-slave-c12x krb5kdc[15522](info): commencing operation
krb5kdc: starting...
W20260808 21:00:55.593689 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:55 dist-test-slave-c12x krb5kdc[15522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:00:55.731153 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:39505
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:46227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:39505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:55.817999 15542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:55.818272 15542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:55.818311 15542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:55.819643 15542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:55.819712 15542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:55.819736 15542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:55.819765 15542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:55.819795 15542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:55.821375 15542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:46227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:39505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:39505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:55.821676 15542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:55.821915 15542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:55.827088 15556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:55.827172 15552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:55.827088 15551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:55.827488 15542 server_base.cc:1034] running on GCE node
I20260808 21:00:55.827759 15542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:55.828071 15542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:55.829248 15542 hybrid_clock.cc:648] HybridClock initialized: now 1786222855829234 us; error 42 us; skew 500 ppm
Aug 08 21:00:55 dist-test-slave-c12x krb5kdc[15522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222855, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:55.831836 15542 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:55.832306 15542 webserver.cc:460] Webserver started at http://127.15.232.190:43289/ using document root <none> and password file <none>
I20260808 21:00:55.832551 15542 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:55.832620 15542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:55.832772 15542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:55.833889 15542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "a667e67ba3534c569c7c621a797589f0"
format_stamp: "Formatted at 2026-08-08 21:00:55 on dist-test-slave-c12x"
server_key: "920a752c93581862178f3f2f1b337cd0"
server_key_iv: "9da97bdd71a38351529a47aa085dbd62"
server_key_version: "encryptionkey@0"
I20260808 21:00:55.834272 15542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "a667e67ba3534c569c7c621a797589f0"
format_stamp: "Formatted at 2026-08-08 21:00:55 on dist-test-slave-c12x"
server_key: "920a752c93581862178f3f2f1b337cd0"
server_key_iv: "9da97bdd71a38351529a47aa085dbd62"
server_key_version: "encryptionkey@0"
I20260808 21:00:55.835765 15542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 21:00:55.836642 15564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:55.836951 15542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:55.837083 15542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "a667e67ba3534c569c7c621a797589f0"
format_stamp: "Formatted at 2026-08-08 21:00:55 on dist-test-slave-c12x"
server_key: "920a752c93581862178f3f2f1b337cd0"
server_key_iv: "9da97bdd71a38351529a47aa085dbd62"
server_key_version: "encryptionkey@0"
I20260808 21:00:55.837172 15542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:55.855422 15542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:55.858048 15542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:55.858181 15542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:55.862483 15542 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:39505
I20260808 21:00:55.862709 15638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:39505 every 8 connection(s)
I20260808 21:00:55.862946 15542 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:55.864243 15639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:55.866950 15639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0: Bootstrap starting.
I20260808 21:00:55.867743 15639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:55.867983 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 15542
I20260808 21:00:55.868076 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:55.868166 15639 log.cc:824] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:55.868314 16290 external_mini_cluster.cc:1428] Setting key b8205f06b97232483da51505311956fa
I20260808 21:00:55.869199 15639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0: No bootstrap required, opened a new log
I20260808 21:00:55.871924 15639 raft_consensus.cc:348] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a667e67ba3534c569c7c621a797589f0" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39505 } }
I20260808 21:00:55.872109 15639 raft_consensus.cc:374] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:55.872145 15639 raft_consensus.cc:729] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a667e67ba3534c569c7c621a797589f0, State: Initialized, Role: FOLLOWER
I20260808 21:00:55.872288 15639 consensus_queue.cc:260] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a667e67ba3534c569c7c621a797589f0" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39505 } }
I20260808 21:00:55.872385 15639 raft_consensus.cc:388] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:55.872417 15639 raft_consensus.cc:482] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:55.872478 15639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:55.873260 15639 raft_consensus.cc:504] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a667e67ba3534c569c7c621a797589f0" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39505 } }
I20260808 21:00:55.873377 15639 leader_election.cc:302] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a667e67ba3534c569c7c621a797589f0; no voters:
I20260808 21:00:55.873605 15639 leader_election.cc:288] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:55.873672 15650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:55.873901 15650 raft_consensus.cc:686] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [term 1 LEADER]: Becoming Leader. State: Replica: a667e67ba3534c569c7c621a797589f0, State: Running, Role: LEADER
I20260808 21:00:55.873929 15639 sys_catalog.cc:564] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:55.874069 15650 consensus_queue.cc:237] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a667e67ba3534c569c7c621a797589f0" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39505 } }
I20260808 21:00:55.875048 15654 sys_catalog.cc:455] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a667e67ba3534c569c7c621a797589f0. Latest consensus state: current_term: 1 leader_uuid: "a667e67ba3534c569c7c621a797589f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a667e67ba3534c569c7c621a797589f0" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39505 } } }
I20260808 21:00:55.875146 15654 sys_catalog.cc:458] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:55.875367 15653 sys_catalog.cc:455] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a667e67ba3534c569c7c621a797589f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a667e67ba3534c569c7c621a797589f0" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 39505 } } }
I20260808 21:00:55.875423 15653 sys_catalog.cc:458] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0 [sys.catalog]: This master's current role is: LEADER
W20260808 21:00:55.876021 15676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:00:55.876082 15676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 21:00:55.876178 15672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:55.876359 15672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 21:00:55 dist-test-slave-c12x krb5kdc[15522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:00:55.880754 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:55.868882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45157 (local address 127.15.232.190:39505)
0808 21:00:55.869079 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:55.869088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:55.869132 (+ 44us) server_negotiation.cc:407] Connection header received
0808 21:00:55.869725 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:55.869735 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:55.869804 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:55.869976 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:55.870619 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:55.872151 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:55.873286 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:55.874017 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:55.874526 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:55.877236 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:55.877240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:55.877241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:55.877252 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:55.879056 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:55.879530 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:55.879533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:55.879533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:55.879617 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:55.879839 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:55.879839 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:55.879840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:55.880070 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:55.880182 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:55.880326 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 21:00:55.880430 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260808 21:00:55.881798 15672 catalog_manager.cc:1347] Generated new cluster ID: e084c61ae7494de196e0473550f3f0a8
I20260808 21:00:55.881855 15672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:55.882053 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:55.890944 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:55.883465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32831 (local address 127.15.232.190:39505)
0808 21:00:55.883556 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:55.883558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:55.883610 (+ 52us) server_negotiation.cc:407] Connection header received
0808 21:00:55.883689 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:55.883690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:55.883703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:55.883758 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:55.884387 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:55.885150 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:55.886135 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:55.886476 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:55.886554 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:55.887967 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:55.887969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:55.887969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:00:55.887977 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:55.889643 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:55.890036 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:55.890037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:55.890038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:55.890104 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:00:55.890297 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:00:55.890298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:00:55.890298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:00:55.890437 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:55.890567 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:55.890650 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 21:00:55.890735 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 21:00:55.894523 15672 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:55.895255 15672 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:55.899829 15672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a667e67ba3534c569c7c621a797589f0: Generated new TSK 0
I20260808 21:00:55.900084 15672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:55.902982 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 15542
2026-08-08T21:00:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:55 dist-test-slave-c12x krb5kdc[15692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:55 dist-test-slave-c12x krb5kdc[15692](info): set up 2 sockets
Aug 08 21:00:55 dist-test-slave-c12x krb5kdc[15692](info): commencing operation
krb5kdc: starting...
W20260808 21:00:57.241858 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:57 dist-test-slave-c12x krb5kdc[15692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:00:57.390156 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:42191
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:42191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:57.476027 15710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:57.476310 15710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:57.476353 15710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:57.477556 15710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:57.477634 15710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:57.477659 15710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:57.477681 15710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:57.477711 15710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:57.479271 15710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:42191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:42191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:57.479586 15710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:57.479828 15710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:57.485246 15720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:57.485199 15717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:57.485248 15710 server_base.cc:1034] running on GCE node
W20260808 21:00:57.485210 15718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:57.485773 15710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:57.486136 15710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:57.487305 15710 hybrid_clock.cc:648] HybridClock initialized: now 1786222857487259 us; error 62 us; skew 500 ppm
Aug 08 21:00:57 dist-test-slave-c12x krb5kdc[15692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222857, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:57.490212 15710 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:57.490682 15710 webserver.cc:460] Webserver started at http://127.15.232.190:40433/ using document root <none> and password file <none>
I20260808 21:00:57.490916 15710 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:57.490965 15710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:57.491138 15710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:57.492347 15710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "d9e0dc2d309e4735a15261e286c2b3d1"
format_stamp: "Formatted at 2026-08-08 21:00:57 on dist-test-slave-c12x"
server_key: "f1fb35dc45aae72def479e23eda0f4e3"
server_key_iv: "ba01ddc291198115584b0b004ed262e5"
server_key_version: "encryptionkey@0"
I20260808 21:00:57.492735 15710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "d9e0dc2d309e4735a15261e286c2b3d1"
format_stamp: "Formatted at 2026-08-08 21:00:57 on dist-test-slave-c12x"
server_key: "f1fb35dc45aae72def479e23eda0f4e3"
server_key_iv: "ba01ddc291198115584b0b004ed262e5"
server_key_version: "encryptionkey@0"
I20260808 21:00:57.494174 15710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 21:00:57.495057 15733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:57.495280 15710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:57.495425 15710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "d9e0dc2d309e4735a15261e286c2b3d1"
format_stamp: "Formatted at 2026-08-08 21:00:57 on dist-test-slave-c12x"
server_key: "f1fb35dc45aae72def479e23eda0f4e3"
server_key_iv: "ba01ddc291198115584b0b004ed262e5"
server_key_version: "encryptionkey@0"
I20260808 21:00:57.495564 15710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:57.535048 15710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:57.537523 15710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:57.537698 15710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:57.541888 15710 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:42191
I20260808 21:00:57.541946 15809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:42191 every 8 connection(s)
I20260808 21:00:57.542325 15710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:57.543284 15813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:57.545809 15813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1: Bootstrap starting.
I20260808 21:00:57.546399 15813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:57.546643 15813 log.cc:824] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:57.547077 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 15710
I20260808 21:00:57.547217 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:57.547410 16290 external_mini_cluster.cc:1428] Setting key dbd11ff66f80cd07c56db409c78adec9
I20260808 21:00:57.547551 15813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1: No bootstrap required, opened a new log
I20260808 21:00:57.550761 15813 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e0dc2d309e4735a15261e286c2b3d1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42191 } }
I20260808 21:00:57.550951 15813 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:57.550982 15813 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9e0dc2d309e4735a15261e286c2b3d1, State: Initialized, Role: FOLLOWER
I20260808 21:00:57.551106 15813 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e0dc2d309e4735a15261e286c2b3d1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42191 } }
I20260808 21:00:57.551208 15813 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:57.551244 15813 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:57.551293 15813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:57.552014 15813 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e0dc2d309e4735a15261e286c2b3d1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42191 } }
I20260808 21:00:57.552127 15813 leader_election.cc:302] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9e0dc2d309e4735a15261e286c2b3d1; no voters:
I20260808 21:00:57.552306 15813 leader_election.cc:288] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:57.552366 15818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:57.552567 15813 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:57.552589 15818 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [term 1 LEADER]: Becoming Leader. State: Replica: d9e0dc2d309e4735a15261e286c2b3d1, State: Running, Role: LEADER
I20260808 21:00:57.552685 15818 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e0dc2d309e4735a15261e286c2b3d1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42191 } }
I20260808 21:00:57.553164 15819 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9e0dc2d309e4735a15261e286c2b3d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e0dc2d309e4735a15261e286c2b3d1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42191 } } }
I20260808 21:00:57.553287 15819 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:57.553437 15818 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9e0dc2d309e4735a15261e286c2b3d1. Latest consensus state: current_term: 1 leader_uuid: "d9e0dc2d309e4735a15261e286c2b3d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9e0dc2d309e4735a15261e286c2b3d1" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 42191 } } }
I20260808 21:00:57.553509 15818 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:57.554364 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:57.548229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38391 (local address 127.15.232.190:42191)
0808 21:00:57.548464 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:57.548474 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:57.548516 (+ 42us) server_negotiation.cc:407] Connection header received
0808 21:00:57.549212 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:57.549222 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:57.549298 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:57.549465 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:57.549660 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:57.551087 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:57.552022 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:57.552681 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:57.552844 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:57.553613 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:57.553617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:57.553617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:57.553834 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:57.553875 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:57.553892 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:57.553955 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:57.554083 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 21:00:57.554162 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":84,"threads_started":1}
I20260808 21:00:57.554410 15848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:57.554980 15848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:57.555527 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:57.556787 15848 catalog_manager.cc:1347] Generated new cluster ID: 52e195ba1bdf4737b6e87dc987b7ddaa
I20260808 21:00:57.556843 15848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:57.560577 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:57.557012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36631 (local address 127.15.232.190:42191)
0808 21:00:57.557057 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:57.557059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:57.557191 (+ 132us) server_negotiation.cc:407] Connection header received
0808 21:00:57.557249 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:57.557249 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:57.557260 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:57.557318 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:57.557447 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:57.558172 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:57.559193 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:00:57.559517 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:00:57.559571 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:00:57.560338 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:57.560339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:57.560339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:57.560344 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:57.560358 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:57.560362 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:57.560387 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:57.560475 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 21:00:57.560499 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 21:00:57.570281 15848 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:57.571133 15848 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:57.579030 15848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9e0dc2d309e4735a15261e286c2b3d1: Generated new TSK 0
I20260808 21:00:57.579277 15848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:57.582897 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 15710
2026-08-08T21:00:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1678 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:57 dist-test-slave-c12x krb5kdc[15863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:57 dist-test-slave-c12x krb5kdc[15863](info): set up 2 sockets
Aug 08 21:00:57 dist-test-slave-c12x krb5kdc[15863](info): commencing operation
krb5kdc: starting...
W20260808 21:00:58.853981 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:00:58 dist-test-slave-c12x krb5kdc[15863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:00:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:00:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:00:58.998946 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37775
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:37775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:00:59.094617 15884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:00:59.094924 15884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:00:59.094970 15884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:00:59.096254 15884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:00:59.096334 15884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:00:59.096361 15884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:00:59.096397 15884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:00:59.096428 15884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:00:59.098043 15884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:37775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:00:59.098356 15884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:00:59.098609 15884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:00:59.104341 15892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:59.104557 15884 server_base.cc:1034] running on GCE node
W20260808 21:00:59.104341 15898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:00:59.104442 15893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:00:59.105008 15884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:00:59.105362 15884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:00:59.106567 15884 hybrid_clock.cc:648] HybridClock initialized: now 1786222859106544 us; error 41 us; skew 500 ppm
Aug 08 21:00:59 dist-test-slave-c12x krb5kdc[15863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222859, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:00:59.109534 15884 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:00:59.110141 15884 webserver.cc:460] Webserver started at http://127.15.232.190:36357/ using document root <none> and password file <none>
I20260808 21:00:59.110400 15884 fs_manager.cc:359] Metadata directory not provided
I20260808 21:00:59.110472 15884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:00:59.110637 15884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:00:59.111843 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "9af681e95a014130b1b967d5b537a7d9"
format_stamp: "Formatted at 2026-08-08 21:00:59 on dist-test-slave-c12x"
server_key: "1f17575ad6223835f76385523daf0a61"
server_key_iv: "f09d39ed4176e7db2ea86d52c2eb3515"
server_key_version: "encryptionkey@0"
I20260808 21:00:59.112229 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "9af681e95a014130b1b967d5b537a7d9"
format_stamp: "Formatted at 2026-08-08 21:00:59 on dist-test-slave-c12x"
server_key: "1f17575ad6223835f76385523daf0a61"
server_key_iv: "f09d39ed4176e7db2ea86d52c2eb3515"
server_key_version: "encryptionkey@0"
I20260808 21:00:59.113725 15884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:00:59.114603 15909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:00:59.114840 15884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:00:59.114967 15884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "9af681e95a014130b1b967d5b537a7d9"
format_stamp: "Formatted at 2026-08-08 21:00:59 on dist-test-slave-c12x"
server_key: "1f17575ad6223835f76385523daf0a61"
server_key_iv: "f09d39ed4176e7db2ea86d52c2eb3515"
server_key_version: "encryptionkey@0"
I20260808 21:00:59.115056 15884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:00:59.118588 15884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:00:59.121060 15884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:00:59.121229 15884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:00:59.125979 15884 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:37775
I20260808 21:00:59.126019 15988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:37775 every 8 connection(s)
I20260808 21:00:59.126389 15884 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:00:59.127642 15989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:00:59.130081 15989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9: Bootstrap starting.
I20260808 21:00:59.130668 15989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9: Neither blocks nor log segments found. Creating new log.
I20260808 21:00:59.130911 15989 log.cc:824] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9: Log is configured to *not* fsync() on all Append() calls
I20260808 21:00:59.131757 15989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9: No bootstrap required, opened a new log
I20260808 21:00:59.133391 15989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af681e95a014130b1b967d5b537a7d9" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37775 } }
I20260808 21:00:59.133555 15989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 21:00:59.133581 15989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9af681e95a014130b1b967d5b537a7d9, State: Initialized, Role: FOLLOWER
I20260808 21:00:59.133734 15989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af681e95a014130b1b967d5b537a7d9" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37775 } }
I20260808 21:00:59.133821 15989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 21:00:59.133845 15989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 21:00:59.133875 15989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:00:59.134761 15989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af681e95a014130b1b967d5b537a7d9" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37775 } }
I20260808 21:00:59.134862 15989 leader_election.cc:302] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9af681e95a014130b1b967d5b537a7d9; no voters:
I20260808 21:00:59.135012 15989 leader_election.cc:288] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 21:00:59.135113 15995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:00:59.135282 15995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [term 1 LEADER]: Becoming Leader. State: Replica: 9af681e95a014130b1b967d5b537a7d9, State: Running, Role: LEADER
I20260808 21:00:59.135308 15989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:00:59.135607 15995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af681e95a014130b1b967d5b537a7d9" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37775 } }
I20260808 21:00:59.136149 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 15884
I20260808 21:00:59.136142 15998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9af681e95a014130b1b967d5b537a7d9. Latest consensus state: current_term: 1 leader_uuid: "9af681e95a014130b1b967d5b537a7d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af681e95a014130b1b967d5b537a7d9" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37775 } } }
I20260808 21:00:59.136227 15998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:59.136268 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:00:59.136479 16290 external_mini_cluster.cc:1428] Setting key 353d7d70fc08121fdd49af781785204b
I20260808 21:00:59.136612 15997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9af681e95a014130b1b967d5b537a7d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9af681e95a014130b1b967d5b537a7d9" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37775 } } }
I20260808 21:00:59.136691 15997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9 [sys.catalog]: This master's current role is: LEADER
I20260808 21:00:59.136857 16007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:00:59.137408 16007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:00:59.139005 16007 catalog_manager.cc:1347] Generated new cluster ID: d2b39f8d9d774377b01ffc66dd0d33e9
I20260808 21:00:59.139060 16007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:00:59.140192 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:59.136986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53593 (local address 127.15.232.190:37775)
0808 21:00:59.137256 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:59.137267 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:59.138229 (+ 962us) server_negotiation.cc:407] Connection header received
0808 21:00:59.138397 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:59.138407 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:59.138440 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:59.138586 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:59.139644 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:59.139647 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:59.139648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:59.139653 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:59.139699 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:59.139713 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:59.139759 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:59.139956 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 21:00:59.139995 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":103,"threads_started":1}
I20260808 21:00:59.141093 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:00:59.144860 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:00:59.142612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41841 (local address 127.15.232.190:37775)
0808 21:00:59.142645 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 21:00:59.142647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:00:59.143713 (+ 1066us) server_negotiation.cc:407] Connection header received
0808 21:00:59.143809 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:00:59.143810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:00:59.143816 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:00:59.143844 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:00:59.144628 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:00:59.144629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:00:59.144630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 21:00:59.144634 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:00:59.144648 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 21:00:59.144654 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:00:59.144681 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 21:00:59.144775 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 21:00:59.144791 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 21:00:59.153828 16007 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:00:59.154036 16007 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:00:59.162942 16007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9af681e95a014130b1b967d5b537a7d9: Generated new TSK 0
I20260808 21:00:59.163187 16007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 21:00:59.174954 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 15884
2026-08-08T21:00:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1596 ms)
[----------] 18 tests from AuthTokenIssuingTest (28805 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:00:59 dist-test-slave-c12x krb5kdc[16035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:00:59 dist-test-slave-c12x krb5kdc[16035](info): set up 2 sockets
Aug 08 21:00:59 dist-test-slave-c12x krb5kdc[16035](info): commencing operation
krb5kdc: starting...
W20260808 21:01:00.474910 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:01:00 dist-test-slave-c12x krb5kdc[16035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:01:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:01:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:01:00.618686 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:34185
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:34185,127.15.232.189:40967,127.15.232.188:36189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:01:00.714119 16059 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 21:01:00.714367 16059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:01:00.714386 16059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:01:00.714411 16059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:01:00.715898 16059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:01:00.715957 16059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:01:00.715973 16059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:01:00.715986 16059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:01:00.716158 16059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:01:00.718255 16059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:34185,127.15.232.189:40967,127.15.232.188:36189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:34185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:01:00.718616 16059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:01:00.718899 16059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:01:00.724696 16072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:00.724694 16066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:01:00.724875 16059 server_base.cc:1034] running on GCE node
W20260808 21:01:00.725028 16069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:01:00.725327 16059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:01:00.725693 16059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:01:00.726878 16059 hybrid_clock.cc:648] HybridClock initialized: now 1786222860726853 us; error 45 us; skew 500 ppm
Aug 08 21:01:00 dist-test-slave-c12x krb5kdc[16035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:01:00.729985 16059 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:01:00.730496 16059 webserver.cc:460] Webserver started at http://127.15.232.190:33579/ using document root <none> and password file <none>
I20260808 21:01:00.730756 16059 fs_manager.cc:359] Metadata directory not provided
I20260808 21:01:00.730829 16059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:01:00.731001 16059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:01:00.732223 16059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "44bcf947d8894a5a96e6792de31dc815"
format_stamp: "Formatted at 2026-08-08 21:01:00 on dist-test-slave-c12x"
server_key: "7b7a084d755d1965392bcc6c4cc46c94"
server_key_iv: "65d85b38a1a009af6968dbdde844e138"
server_key_version: "encryptionkey@0"
I20260808 21:01:00.732614 16059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "44bcf947d8894a5a96e6792de31dc815"
format_stamp: "Formatted at 2026-08-08 21:01:00 on dist-test-slave-c12x"
server_key: "7b7a084d755d1965392bcc6c4cc46c94"
server_key_iv: "65d85b38a1a009af6968dbdde844e138"
server_key_version: "encryptionkey@0"
I20260808 21:01:00.734098 16059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:01:00.735170 16082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:01:00.735626 16059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:01:00.735726 16059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "44bcf947d8894a5a96e6792de31dc815"
format_stamp: "Formatted at 2026-08-08 21:01:00 on dist-test-slave-c12x"
server_key: "7b7a084d755d1965392bcc6c4cc46c94"
server_key_iv: "65d85b38a1a009af6968dbdde844e138"
server_key_version: "encryptionkey@0"
I20260808 21:01:00.735853 16059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:01:00.745708 16059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:01:00.748313 16059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:01:00.748443 16059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:01:00.753285 16059 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:34185
I20260808 21:01:00.753722 16059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:01:00.753701 16162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:34185 every 8 connection(s)
I20260808 21:01:00.754617 16163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:01:00.755728 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16059
I20260808 21:01:00.755883 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:01:00.756106 16290 external_mini_cluster.cc:1428] Setting key 515022675f77334f1301e64666ee46be
I20260808 21:01:00.756703 16163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 21:01:00 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.189@KRBTEST.COM" created.
I20260808 21:01:00.768612 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.757649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56470 (local address 127.15.232.190:34185)
0808 21:01:00.757878 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:00.757883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:00.758005 (+ 122us) server_negotiation.cc:407] Connection header received
0808 21:01:00.758687 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:00.758689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:00.758719 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:00.758823 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:00.759489 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.760451 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.761518 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.761879 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.762047 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:00.764509 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:00.764517 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:00.764518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:00.764526 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:00.766489 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:00.766973 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:00.766977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:00.766977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:00.767076 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:00.767357 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:00.767358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:00.767358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:00.767764 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:00.767876 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:00.768098 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 21:01:00.768312 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":131,"threads_started":1}
I20260808 21:01:00.768611 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.757576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:34185 (local address 127.0.0.1:56470)
0808 21:01:00.757797 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.757851 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:00.758488 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:00.758828 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:00.758830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:00.759165 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:00.759339 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:00.759344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.760585 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.760586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:00.761410 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:00.761413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.762005 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.762006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:00.762085 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:00.762737 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:00.762748 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:00.764396 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:00.766613 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:00.766622 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:00.766625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:00.766854 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:00.767157 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:00.767158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:00.767159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:00.767262 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:00.767851 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:00.767857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:00.767930 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 21:01:00.768057 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 21:01:00.768390 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":158,"thread_start_us":108,"threads_started":1}
I20260808 21:01:00.769683 16163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } has no permanent_uuid. Determining permanent_uuid...
I20260808 21:01:00.770117 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.769975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:40967 (local address 127.0.0.1:48874)
0808 21:01:00.770034 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.770089 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 21:01:00.770319 16084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
I20260808 21:01:00.770871 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.770743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:40967 (local address 127.0.0.1:48876)
0808 21:01:00.770813 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.770848 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 21:01:00.771041 16163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.189:40967: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab.
Entry for principal kudu/127.15.232.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab.
I20260808 21:01:00.789188 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.189:40967
--webserver_interface=127.15.232.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab
--principal=kudu/127.15.232.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:34185,127.15.232.189:40967,127.15.232.188:36189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 21:01:00.820344 16163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } attempt: 1
I20260808 21:01:00.821008 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.820805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:40967 (local address 127.0.0.1:48888)
0808 21:01:00.820901 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.820963 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 21:01:00.821503 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.821383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:40967 (local address 127.0.0.1:48900)
0808 21:01:00.821439 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.821477 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 21:01:00.821676 16163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.189:40967: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
W20260808 21:01:00.877964 16171 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 21:01:00.878225 16171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:01:00.878245 16171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:01:00.878269 16171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:01:00.879588 16171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:01:00.879647 16171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:01:00.879663 16171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:01:00.879678 16171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:01:00.879693 16171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:01:00.881304 16171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:34185,127.15.232.189:40967,127.15.232.188:36189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab
--principal=kudu/127.15.232.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.189:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.232.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:01:00.881618 16171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:01:00.881886 16171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:01:00.887284 16181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:00.887279 16178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:00.887277 16183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:01:00.887774 16171 server_base.cc:1034] running on GCE node
I20260808 21:01:00.888130 16171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:01:00.888484 16171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:01:00.889694 16171 hybrid_clock.cc:648] HybridClock initialized: now 1786222860889668 us; error 58 us; skew 500 ppm
I20260808 21:01:00.889920 16163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } attempt: 2
I20260808 21:01:00.890403 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.890241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:40967 (local address 127.0.0.1:48912)
0808 21:01:00.890316 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.890358 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 21:01:00.890841 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.890728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:40967 (local address 127.0.0.1:48924)
0808 21:01:00.890781 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.890815 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 21:01:00.890957 16163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.189:40967: Network error: Client connection negotiation failed: client connection to 127.15.232.189:40967: connect: Connection refused (error 111)
Aug 08 21:01:00 dist-test-slave-c12x krb5kdc[16035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:01:00.892789 16171 init.cc:377] Logged in from keytab as kudu/127.15.232.189@KRBTEST.COM (short username kudu)
I20260808 21:01:00.893249 16171 webserver.cc:460] Webserver started at http://127.15.232.189:46617/ using document root <none> and password file <none>
I20260808 21:01:00.893487 16171 fs_manager.cc:359] Metadata directory not provided
I20260808 21:01:00.893558 16171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:01:00.893716 16171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:01:00.894889 16171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/data/instance:
uuid: "42749c7108184706be1fff1504531b6c"
format_stamp: "Formatted at 2026-08-08 21:01:00 on dist-test-slave-c12x"
server_key: "4da1ccb2c6023954e32b49fb153ce688"
server_key_iv: "a102f91041e29347c2568eb18af84140"
server_key_version: "encryptionkey@0"
I20260808 21:01:00.895283 16171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/wal/instance:
uuid: "42749c7108184706be1fff1504531b6c"
format_stamp: "Formatted at 2026-08-08 21:01:00 on dist-test-slave-c12x"
server_key: "4da1ccb2c6023954e32b49fb153ce688"
server_key_iv: "a102f91041e29347c2568eb18af84140"
server_key_version: "encryptionkey@0"
I20260808 21:01:00.896835 16171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:01:00.897655 16191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:01:00.897902 16171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 21:01:00.898030 16171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/wal
uuid: "42749c7108184706be1fff1504531b6c"
format_stamp: "Formatted at 2026-08-08 21:01:00 on dist-test-slave-c12x"
server_key: "4da1ccb2c6023954e32b49fb153ce688"
server_key_iv: "a102f91041e29347c2568eb18af84140"
server_key_version: "encryptionkey@0"
I20260808 21:01:00.898114 16171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:01:00.938470 16171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:01:00.941184 16171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:01:00.941381 16171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:01:00.946420 16171 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.189:40967
I20260808 21:01:00.946455 16270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.189:40967 every 8 connection(s)
I20260808 21:01:00.946888 16171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
I20260808 21:01:00.947896 16271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:01:00.949618 16271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 21:01:00 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:01:00.956547 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16171
I20260808 21:01:00.956658 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-1/wal/instance
I20260808 21:01:00.956872 16290 external_mini_cluster.cc:1428] Setting key 678be698ec28137ec90163d13f16cca2
I20260808 21:01:00.960671 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.950566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56482 (local address 127.15.232.190:34185)
0808 21:01:00.950641 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:00.950643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:00.950908 (+ 265us) server_negotiation.cc:407] Connection header received
0808 21:01:00.951474 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:00.951477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:00.951498 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:00.951568 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:00.952252 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.952801 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.954055 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.954272 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.954333 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:00.957052 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:00.957054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:00.957054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:00.957064 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:00.958884 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:00.959552 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:00.959553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:00.959553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:00.959613 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:00.959958 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:00.959959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:00.959959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:00.960097 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:00.960150 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:00.960522 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 21:01:00.960600 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 21:01:00.961004 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.950532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:34185 (local address 127.0.0.1:56482)
0808 21:01:00.950742 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.950786 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:00.951343 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:00.951642 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:00.951646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:00.951938 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:00.952147 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:00.952151 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.952922 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.952923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:00.953959 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:00.953962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.954413 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.954414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:00.954552 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:00.955159 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:00.955168 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:00.956864 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:00.959009 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:00.959022 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:00.959025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:00.959407 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:00.959688 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:00.959689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:00.959689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:00.959887 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:00.960168 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:00.960174 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:00.960373 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 21:01:00.960502 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 21:01:00.960695 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":127,"threads_started":1}
I20260808 21:01:00.961666 16271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 21:01:00 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.188@KRBTEST.COM" created.
I20260808 21:01:00.970754 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.961959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:40967 (local address 127.0.0.1:48940)
0808 21:01:00.962008 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.962019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:00.962057 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:00.962384 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:00.962385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:00.962555 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:00.962627 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:00.962629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.963646 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.963647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:00.964536 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:00.964541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.965040 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.965042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:00.965127 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:00.965582 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:00.965587 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:00.967074 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:00.969393 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:00.969394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:00.969394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:00.969577 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:00.969828 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:00.969830 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:00.969830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:00.969905 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:00.970252 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:00.970253 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:00.970295 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 21:01:00.970493 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 21:01:00.970629 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 21:01:00.970829 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.962037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48940 (local address 127.15.232.189:40967)
0808 21:01:00.962214 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:00.962219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:00.962244 (+ 25us) server_negotiation.cc:407] Connection header received
0808 21:01:00.962263 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:00.962265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:00.962284 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:00.962340 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:00.962763 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.963546 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.964633 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.964953 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.965016 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:00.967174 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:00.967187 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:00.967188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:00.967195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:00.969245 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:00.969684 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:00.969688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:00.969689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:00.969762 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:00.969972 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:00.969973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:00.969973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:00.970148 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:00.970245 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:00.970545 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 21:01:00.970690 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260808 21:01:00.971791 16271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } has no permanent_uuid. Determining permanent_uuid...
I20260808 21:01:00.972229 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.972081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39752)
0808 21:01:00.972134 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.972192 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 21:01:00.972417 16196 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
I20260808 21:01:00.972973 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.972856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39756)
0808 21:01:00.972910 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.972942 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 21:01:00.973186 16271 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:36189: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
I20260808 21:01:00.982170 16163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } attempt: 3
Aug 08 21:01:00 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab.
Entry for principal kudu/127.15.232.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab.
I20260808 21:01:00.991221 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.188:36189
--webserver_interface=127.15.232.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab
--principal=kudu/127.15.232.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:40271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:34185,127.15.232.189:40967,127.15.232.188:36189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 21:01:00.991891 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.982915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.15.232.189:40967)
0808 21:01:00.982991 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:00.982994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:00.983005 (+ 11us) server_negotiation.cc:407] Connection header received
0808 21:01:00.983039 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:00.983041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:00.983057 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:00.983131 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:00.983558 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.984290 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.985406 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.985700 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.985794 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:00.988425 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:00.988427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:00.988428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:00.988436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:00.990309 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:00.990826 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:00.990827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:00.990828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:00.990909 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:00.991117 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:00.991119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:00.991119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:00.991288 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:00.991397 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:00.991745 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 21:01:00.991823 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 21:01:00.991943 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.982725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:40967 (local address 127.0.0.1:48944)
0808 21:01:00.982853 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.982870 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:00.982941 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:00.983146 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:00.983147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:00.983308 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:00.983377 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:00.983378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.984457 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.984458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:00.985291 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:00.985295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:00.985821 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:00.985822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:00.985903 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:00.986605 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:00.986611 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:00.988307 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:00.990422 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:00.990423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:00.990424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:00.990729 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:00.990987 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:00.990988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:00.990988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:00.991048 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:00.991527 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:00.991529 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:00.991586 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 21:01:00.991664 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 21:01:00.991784 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 21:01:00.992465 16163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } has no permanent_uuid. Determining permanent_uuid...
I20260808 21:01:00.992870 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.992738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39762)
0808 21:01:00.992793 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.992833 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 21:01:00.993227 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:00.993117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39772)
0808 21:01:00.993166 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:00.993194 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 21:01:00.993321 16163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:36189: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
I20260808 21:01:01.022514 16271 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } attempt: 1
I20260808 21:01:01.023242 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.023090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39782)
0808 21:01:01.023162 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.023207 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 21:01:01.023878 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.023717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39790)
0808 21:01:01.023819 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.023854 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260808 21:01:01.024025 16271 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:36189: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
I20260808 21:01:01.024484 16163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } attempt: 1
I20260808 21:01:01.024997 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.024870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39794)
0808 21:01:01.024930 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.024969 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 21:01:01.025375 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.025275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39800)
0808 21:01:01.025321 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.025355 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 21:01:01.025465 16163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:36189: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
W20260808 21:01:01.084617 16285 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 21:01:01.084894 16285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:01:01.084916 16285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:01:01.084942 16285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:01:01.086323 16285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:01:01.086400 16285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:01:01.086416 16285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:01:01.086431 16285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:01:01.086445 16285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:01:01.088341 16285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:40271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:34185,127.15.232.189:40967,127.15.232.188:36189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab
--principal=kudu/127.15.232.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.188:36189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.232.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:01:01.088681 16285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:01:01.088953 16285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 21:01:01.092222 16271 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } attempt: 2
I20260808 21:01:01.092767 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.092609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39808)
0808 21:01:01.092690 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.092736 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 21:01:01.093160 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.093067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39820)
0808 21:01:01.093106 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.093133 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 21:01:01.093333 16271 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:36189: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
W20260808 21:01:01.094635 16297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:01.094641 16298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:01.094640 16305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:01:01.094828 16285 server_base.cc:1034] running on GCE node
I20260808 21:01:01.095260 16285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:01:01.095705 16285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:01:01.096916 16285 hybrid_clock.cc:648] HybridClock initialized: now 1786222861096902 us; error 32 us; skew 500 ppm
Aug 08 21:01:01 dist-test-slave-c12x krb5kdc[16035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222861, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:01:01.100020 16285 init.cc:377] Logged in from keytab as kudu/127.15.232.188@KRBTEST.COM (short username kudu)
I20260808 21:01:01.100533 16285 webserver.cc:460] Webserver started at http://127.15.232.188:41895/ using document root <none> and password file <none>
I20260808 21:01:01.100611 16163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } attempt: 2
I20260808 21:01:01.100819 16285 fs_manager.cc:359] Metadata directory not provided
I20260808 21:01:01.100948 16285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:01:01.101135 16285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:01:01.101159 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.100981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39834)
0808 21:01:01.101043 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.101121 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 21:01:01.101596 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.101446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39844)
0808 21:01:01.101528 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.101570 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 21:01:01.101711 16163 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:36189: Network error: Client connection negotiation failed: client connection to 127.15.232.188:36189: connect: Connection refused (error 111)
I20260808 21:01:01.102573 16285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/data/instance:
uuid: "02568a128aee41ef8445e3e67abd8ae6"
format_stamp: "Formatted at 2026-08-08 21:01:01 on dist-test-slave-c12x"
server_key: "9f054d51708529ab1a0914211ff65277"
server_key_iv: "5690fd9345d89a4e795f37730d6aa2b4"
server_key_version: "encryptionkey@0"
I20260808 21:01:01.102984 16285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/wal/instance:
uuid: "02568a128aee41ef8445e3e67abd8ae6"
format_stamp: "Formatted at 2026-08-08 21:01:01 on dist-test-slave-c12x"
server_key: "9f054d51708529ab1a0914211ff65277"
server_key_iv: "5690fd9345d89a4e795f37730d6aa2b4"
server_key_version: "encryptionkey@0"
I20260808 21:01:01.104671 16285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 21:01:01.105664 16316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:01:01.105914 16285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:01:01.106052 16285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/wal
uuid: "02568a128aee41ef8445e3e67abd8ae6"
format_stamp: "Formatted at 2026-08-08 21:01:01 on dist-test-slave-c12x"
server_key: "9f054d51708529ab1a0914211ff65277"
server_key_iv: "5690fd9345d89a4e795f37730d6aa2b4"
server_key_version: "encryptionkey@0"
I20260808 21:01:01.106145 16285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:01:01.133899 16285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:01:01.136641 16285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:01:01.136847 16285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:01:01.142056 16285 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.188:36189
I20260808 21:01:01.142132 16402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.188:36189 every 8 connection(s)
I20260808 21:01:01.142550 16285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
I20260808 21:01:01.143827 16403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:01:01.146432 16403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } has no permanent_uuid. Determining permanent_uuid...
I20260808 21:01:01.148677 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16285
I20260808 21:01:01.148814 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786222573557010-16290-0/minicluster-data/master-2/wal/instance
I20260808 21:01:01.149055 16290 external_mini_cluster.cc:1428] Setting key b52f677b5aaf038130233e0b35dc785d
Aug 08 21:01:01 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222861, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
Aug 08 21:01:01 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
I20260808 21:01:01.159614 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.147492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56484 (local address 127.15.232.190:34185)
0808 21:01:01.147560 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.147562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.148060 (+ 498us) server_negotiation.cc:407] Connection header received
0808 21:01:01.148729 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.148731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.148746 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.148801 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.149990 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.150752 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.152262 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.152592 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.152686 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.155671 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.155673 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.155673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.155691 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.157885 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.158435 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.158437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.158438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.158508 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.158809 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.158810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.158811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.158958 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.159038 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.159371 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.159499 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 21:01:01.159821 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.147456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:34185 (local address 127.0.0.1:56484)
0808 21:01:01.147758 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.147796 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:01.148547 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:01.148886 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:01.148892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:01.149345 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:01.149773 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.149781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.150884 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.150885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.152146 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.152150 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.152710 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.152711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.152834 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.153517 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:01.153526 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:01.155555 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:01.158025 (+ 2470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.158037 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.158040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.158327 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.158601 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.158602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.158602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.158728 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.159044 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:01.159053 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:01.159223 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 21:01:01.159333 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 21:01:01.159553 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":130,"threads_started":1}
I20260808 21:01:01.160593 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.149661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.15.232.188:36189)
0808 21:01:01.149903 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.149909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.149936 (+ 27us) server_negotiation.cc:407] Connection header received
0808 21:01:01.150130 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.150137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.150166 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.150239 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.151078 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.152289 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.153219 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.153648 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.153707 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.156437 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.156451 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.156451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.156462 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.158747 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.159162 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.159165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.159166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.159232 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.159475 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.159476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.159476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.159631 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.159825 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.160346 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.160446 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":140,"threads_started":1}
I20260808 21:01:01.160837 16403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 21:01:01 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222861, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
Aug 08 21:01:01 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:01:01.169701 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.161243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48954 (local address 127.15.232.189:40967)
0808 21:01:01.161309 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.161311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.161341 (+ 30us) server_negotiation.cc:407] Connection header received
0808 21:01:01.161371 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.161372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.161384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.161496 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.161890 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.162486 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.163320 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.163596 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.163656 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.166529 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.166530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.166530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.166539 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.168282 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.168759 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.168760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.168761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.168831 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.169077 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.169078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.169079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.169232 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.169319 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.169502 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.169593 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 21:01:01.169705 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.161155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:40967 (local address 127.0.0.1:48954)
0808 21:01:01.161215 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.161228 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:01.161276 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:01.161506 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:01.161507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:01.161730 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:01.161809 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.161812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.162571 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.162571 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.163228 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.163231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.163669 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.163670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.163748 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.164686 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:01.164692 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:01.166362 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:01.168422 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.168424 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.168424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.168655 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.168921 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.168922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.168923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.168997 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.169311 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:01.169312 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:01.169366 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 21:01:01.169465 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 21:01:01.169588 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 21:01:01.170430 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.161630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.15.232.190:34185)
0808 21:01:01.161666 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.161668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.161675 (+ 7us) server_negotiation.cc:407] Connection header received
0808 21:01:01.161697 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.161698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.161705 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.161758 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.162272 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.162835 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.163863 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.164386 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.164475 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.167340 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.167341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.167342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.167349 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.168954 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.169362 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.169364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.169364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.169427 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.169646 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.169647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.169648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.169766 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.169983 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.170198 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.170304 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 21:01:01.170506 16403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 21:01:01 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222861, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
Aug 08 21:01:01 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
I20260808 21:01:01.179105 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.170866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.15.232.188:36189)
0808 21:01:01.170902 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.170904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.170912 (+ 8us) server_negotiation.cc:407] Connection header received
0808 21:01:01.171012 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.171013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.171027 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.171082 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.171388 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.172133 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.172950 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.173190 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.173282 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.175715 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.175716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.175717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.175726 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.177472 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.178024 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.178026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.178026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.178098 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.178342 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.178343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.178343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.178478 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.178683 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.178863 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.179009 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":18}
I20260808 21:01:01.179105 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.170816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39874)
0808 21:01:01.170859 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.170872 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:01.170915 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:01.171085 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:01.171086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:01.171277 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:01.171325 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.171327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.172224 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.172224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.172845 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.172848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.173272 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.173273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.173334 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.173809 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:01.173814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:01.175600 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:01.177652 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.177654 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.177654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.177903 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.178187 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.178188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.178189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.178264 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.178675 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:01.178676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:01.178725 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 21:01:01.178822 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 21:01:01.178955 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 21:01:01.179963 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.171159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.15.232.189:40967)
0808 21:01:01.171200 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.171203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.171234 (+ 31us) server_negotiation.cc:407] Connection header received
0808 21:01:01.171254 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.171255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.171265 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.171325 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.171861 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.172493 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.173303 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.173544 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.173639 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.176658 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.176660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.176661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.176675 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.178445 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.178810 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.178811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.178812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.178876 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.179091 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.179092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.179092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.179420 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.179629 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.179806 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.179898 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 21:01:01.181216 16403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6: Bootstrap starting.
I20260808 21:01:01.182411 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:01:01.182696 16403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6: Neither blocks nor log segments found. Creating new log.
I20260808 21:01:01.183056 16403 log.cc:824] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6: Log is configured to *not* fsync() on all Append() calls
I20260808 21:01:01.184511 16403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6: No bootstrap required, opened a new log
I20260808 21:01:01.184665 16271 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } attempt: 3
I20260808 21:01:01.183804 16120 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "44bcf947d8894a5a96e6792de31dc815"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56492
I20260808 21:01:01.186810 16403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } }
I20260808 21:01:01.187016 16403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02568a128aee41ef8445e3e67abd8ae6, State: Initialized, Role: FOLLOWER
I20260808 21:01:01.187125 16403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } }
I20260808 21:01:01.187558 16403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:01:01.187527 16411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.187639 16411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 21:01:01.190135 16429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:01:01.190269 16429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 21:01:01 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
I20260808 21:01:01.190236 16285 master_runner.cc:186] Error getting master registration for 127.15.232.190:34185: OK, instance_id { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" instance_seqno: 1786222860751743 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 21:01:01.193814 16285 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 21:01:01.194746 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.185221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39888)
0808 21:01:01.185397 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.185437 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:01.185525 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:01.185945 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:01.185946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:01.186243 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:01.186296 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.186300 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.187059 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.187060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.187952 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.187955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.188829 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.188831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.188905 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.189524 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:01.189529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:01.191209 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:01.193443 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.193444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.193445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.193679 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.193902 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.193903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.193904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.193968 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.194372 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:01.194374 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:01.194429 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 21:01:01.194509 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 21:01:01.194635 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 21:01:01.194770 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.185297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39888 (local address 127.15.232.188:36189)
0808 21:01:01.185437 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.185440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.185497 (+ 57us) server_negotiation.cc:407] Connection header received
0808 21:01:01.185625 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.185626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.185639 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.185926 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.186416 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.186949 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.188069 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.188588 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.188717 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.191329 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.191331 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.191331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.191339 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.193334 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.193767 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.193768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.193769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.193829 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.194060 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.194062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.194062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.194273 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.194365 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.194554 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.194659 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 21:01:01.196801 16271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c: Bootstrap starting.
I20260808 21:01:01.197643 16271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c: Neither blocks nor log segments found. Creating new log.
I20260808 21:01:01.197999 16271 log.cc:824] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c: Log is configured to *not* fsync() on all Append() calls
I20260808 21:01:01.199012 16271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c: No bootstrap required, opened a new log
I20260808 21:01:01.200856 16163 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } attempt: 3
I20260808 21:01:01.201388 16271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } }
I20260808 21:01:01.201606 16271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42749c7108184706be1fff1504531b6c, State: Initialized, Role: FOLLOWER
I20260808 21:01:01.201709 16271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } }
I20260808 21:01:01.202104 16433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.202195 16271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:01:01.202279 16433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [sys.catalog]: This master's current role is: FOLLOWER
W20260808 21:01:01.205642 16450 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:01:01.205711 16450 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 21:01:01 dist-test-slave-c12x krb5kdc[16035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
I20260808 21:01:01.205353 16171 master_runner.cc:186] Error getting master registration for 127.15.232.190:34185: OK, instance_id { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" instance_seqno: 1786222860751743 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 21:01:01.208866 16171 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 21:01:01.210878 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.201211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:36189 (local address 127.0.0.1:39898)
0808 21:01:01.201270 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:01.201281 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:01.201324 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:01.201574 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:01.201575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:01.201775 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:01.201831 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.201833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.202954 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.202956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.204061 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:01.204066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.204728 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.204730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:01.204792 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.205341 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:01.205348 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:01.207095 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:01.209357 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.209374 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.209375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.209627 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.210030 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:01.210031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:01.210032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:01.210121 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:01.210473 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:01.210474 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:01.210548 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 21:01:01.210617 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 21:01:01.210756 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 21:01:01.211202 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.201297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39898 (local address 127.15.232.188:36189)
0808 21:01:01.201379 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.201381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.201417 (+ 36us) server_negotiation.cc:407] Connection header received
0808 21:01:01.201449 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.201450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.201463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.201555 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.201939 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.202726 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.204241 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.204565 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.204632 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.207233 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.207235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.207235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.207244 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.209218 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.209725 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.209727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.209727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.209822 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.210196 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.210197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.210198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.210377 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.210448 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.211028 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.211117 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 21:01:01.212970 16163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815: Bootstrap starting.
I20260808 21:01:01.213778 16163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815: Neither blocks nor log segments found. Creating new log.
I20260808 21:01:01.214085 16163 log.cc:824] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815: Log is configured to *not* fsync() on all Append() calls
I20260808 21:01:01.215096 16163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815: No bootstrap required, opened a new log
I20260808 21:01:01.216565 16163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } }
I20260808 21:01:01.216718 16163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44bcf947d8894a5a96e6792de31dc815, State: Initialized, Role: FOLLOWER
I20260808 21:01:01.216810 16163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } }
I20260808 21:01:01.217185 16163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:01:01.217274 16120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 21:01:01.217339 16120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:01:01.218287 16120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } }
I20260808 21:01:01.218595 16120 leader_election.cc:288] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [CANDIDATE]: Term 1 election: Requested vote from peers 42749c7108184706be1fff1504531b6c (127.15.232.189:40967), 02568a128aee41ef8445e3e67abd8ae6 (127.15.232.188:36189)
I20260808 21:01:01.217139 16454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.219130 16454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:01.219154 16365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44bcf947d8894a5a96e6792de31dc815" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "02568a128aee41ef8445e3e67abd8ae6"
I20260808 21:01:01.219319 16365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:01:01.220288 16365 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44bcf947d8894a5a96e6792de31dc815 in term 1.
I20260808 21:01:01.220553 16224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "44bcf947d8894a5a96e6792de31dc815" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "42749c7108184706be1fff1504531b6c"
I20260808 21:01:01.220698 16224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [term 0 FOLLOWER]: Advancing to term 1
W20260808 21:01:01.221246 16469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:01:01.221321 16469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 21:01:01.221577 16224 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44bcf947d8894a5a96e6792de31dc815 in term 1.
I20260808 21:01:01.221874 16086 leader_election.cc:302] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02568a128aee41ef8445e3e67abd8ae6, 44bcf947d8894a5a96e6792de31dc815; no voters:
I20260808 21:01:01.222059 16454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:01:01.222213 16454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [term 1 LEADER]: Becoming Leader. State: Replica: 44bcf947d8894a5a96e6792de31dc815, State: Running, Role: LEADER
I20260808 21:01:01.222311 16454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } }
I20260808 21:01:01.222795 16480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44bcf947d8894a5a96e6792de31dc815. Latest consensus state: current_term: 1 leader_uuid: "44bcf947d8894a5a96e6792de31dc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.222882 16480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [sys.catalog]: This master's current role is: LEADER
I20260808 21:01:01.224310 16059 master_runner.cc:263] No leader master found from master 44bcf947d8894a5a96e6792de31dc815
I20260808 21:01:01.224365 16059 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 21:01:01.227614 16483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:01:01.227841 16483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:01:01.238946 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.223797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.15.232.188:36189)
0808 21:01:01.223842 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.223844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.228781 (+ 4937us) server_negotiation.cc:407] Connection header received
0808 21:01:01.228897 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.228898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.228911 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.228957 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.229845 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.230615 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.231743 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.232047 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.232096 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.233785 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.233787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.233787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.233795 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.235642 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.237841 (+ 2199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.237843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.237843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.237914 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.238230 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.238231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.238232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.238384 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.238483 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.238729 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.238829 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 21:01:01.246627 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.224054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48972 (local address 127.15.232.189:40967)
0808 21:01:01.224096 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.224099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.231617 (+ 7518us) server_negotiation.cc:407] Connection header received
0808 21:01:01.231742 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.231744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.231756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.231791 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.232669 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.233488 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.234775 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.235099 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.235149 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.242092 (+ 6943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.242094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.242094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.242106 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.244158 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.244649 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.244650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.244650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.244718 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.244943 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.244944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.244945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.245084 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.245172 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.246396 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.246512 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 21:01:01.249701 16092 tablet.cc:2369] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 21:01:01.252517 16166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:01.224239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56506 (local address 127.15.232.190:34185)
0808 21:01:01.224281 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.224284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.234721 (+ 10437us) server_negotiation.cc:407] Connection header received
0808 21:01:01.235785 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.235787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.235799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.235834 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.239293 (+ 3459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.240162 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.245402 (+ 5240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:01.245781 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:01.245830 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:01.248469 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:01.248471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:01.248472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:01.248481 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:01.250890 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.251353 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.251355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.251355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.251550 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:01.251795 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:01.251796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:01.251797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:01.251954 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:01.252053 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:01.252292 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 21:01:01.252392 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 21:01:01.252652 16365 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [term 1 FOLLOWER]: Refusing update from remote peer 44bcf947d8894a5a96e6792de31dc815: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:01:01.253108 16480 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:01:01.253765 16224 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [term 1 FOLLOWER]: Refusing update from remote peer 44bcf947d8894a5a96e6792de31dc815: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:01:01.254077 16480 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:01:01.255081 16411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44bcf947d8894a5a96e6792de31dc815. Latest consensus state: current_term: 1 leader_uuid: "44bcf947d8894a5a96e6792de31dc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.255239 16411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:01.255367 16433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44bcf947d8894a5a96e6792de31dc815. Latest consensus state: current_term: 1 leader_uuid: "44bcf947d8894a5a96e6792de31dc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.255486 16433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:01.261010 16480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44bcf947d8894a5a96e6792de31dc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.261142 16480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [sys.catalog]: This master's current role is: LEADER
I20260808 21:01:01.261369 16493 mvcc.cc:205] Tried to move back new op lower bound from 7316368839687196672 to 7316368839567327232. Current Snapshot: MvccSnapshot[applied={T|T < 7316368839687196672}]
I20260808 21:01:01.262523 16433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44bcf947d8894a5a96e6792de31dc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.262648 16433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:01.262889 16496 mvcc.cc:205] Tried to move back new op lower bound from 7316368839687196672 to 7316368839567327232. Current Snapshot: MvccSnapshot[applied={T|T < 7316368839687196672}]
I20260808 21:01:01.264292 16454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44bcf947d8894a5a96e6792de31dc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.264417 16454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815 [sys.catalog]: This master's current role is: LEADER
I20260808 21:01:01.264921 16411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44bcf947d8894a5a96e6792de31dc815" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44bcf947d8894a5a96e6792de31dc815" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 34185 } } peers { permanent_uuid: "42749c7108184706be1fff1504531b6c" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 40967 } } peers { permanent_uuid: "02568a128aee41ef8445e3e67abd8ae6" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 36189 } } }
I20260808 21:01:01.265043 16411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:01.265120 16494 mvcc.cc:205] Tried to move back new op lower bound from 7316368839687196672 to 7316368839567327232. Current Snapshot: MvccSnapshot[applied={T|T < 7316368839687196672}]
I20260808 21:01:01.265224 16483 catalog_manager.cc:1347] Generated new cluster ID: bc97ad6daaf94f7292e7750e716427a0
I20260808 21:01:01.265278 16483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:01:01.283779 16483 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:01:01.284657 16483 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:01:01.291357 16483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44bcf947d8894a5a96e6792de31dc815: Generated new TSK 0
I20260808 21:01:01.291652 16483 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 21:01:01.320019 16408 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.318176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.15.232.188:36189)
0808 21:01:01.318234 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.318236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.319261 (+ 1025us) server_negotiation.cc:407] Connection header received
0808 21:01:01.319378 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.319380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.319392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.319425 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.319948 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39914: BlockingRecv error: recv got EOF from 127.0.0.1:39914 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 21:01:01.320860 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.318589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48978 (local address 127.15.232.189:40967)
0808 21:01:01.318634 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.318636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.320304 (+ 1668us) server_negotiation.cc:407] Connection header received
0808 21:01:01.320411 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.320412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.320423 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.320453 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.320795 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48978: BlockingRecv error: recv got EOF from 127.0.0.1:48978 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 21:01:01.321826 16166 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.318949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56522 (local address 127.15.232.190:34185)
0808 21:01:01.318991 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.318994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.321275 (+ 2281us) server_negotiation.cc:407] Connection header received
0808 21:01:01.321376 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.321377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.321388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.321419 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.321758 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56522: BlockingRecv error: recv got EOF from 127.0.0.1:56522 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 21:01:01.324604 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.323261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48988 (local address 127.15.232.189:40967)
0808 21:01:01.323301 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.323303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.324028 (+ 725us) server_negotiation.cc:407] Connection header received
0808 21:01:01.324162 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.324163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.324183 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.324214 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.324553 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48988: BlockingRecv error: recv got EOF from 127.0.0.1:48988 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 21:01:01.327917 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.327006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48998 (local address 127.15.232.189:40967)
0808 21:01:01.327047 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.327059 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.327319 (+ 260us) server_negotiation.cc:407] Connection header received
0808 21:01:01.327424 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.327425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.327471 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.327504 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.327865 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48998: BlockingRecv error: recv got EOF from 127.0.0.1:48998 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 21:01:01.332183 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.331198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49002 (local address 127.15.232.189:40967)
0808 21:01:01.331239 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.331240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.331609 (+ 369us) server_negotiation.cc:407] Connection header received
0808 21:01:01.331717 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.331718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.331730 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.331763 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.332131 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49002: BlockingRecv error: recv got EOF from 127.0.0.1:49002 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 21:01:01.338486 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.337516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49012 (local address 127.15.232.189:40967)
0808 21:01:01.337559 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.337562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.337825 (+ 263us) server_negotiation.cc:407] Connection header received
0808 21:01:01.337933 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.337935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.337946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.337978 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.338427 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49012: BlockingRecv error: recv got EOF from 127.0.0.1:49012 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 21:01:01.349205 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.348155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.15.232.189:40967)
0808 21:01:01.348206 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.348208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.348527 (+ 319us) server_negotiation.cc:407] Connection header received
0808 21:01:01.348639 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.348640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.348654 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.348686 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.349134 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49024: BlockingRecv error: recv got EOF from 127.0.0.1:49024 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 21:01:01.367743 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.366721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49028 (local address 127.15.232.189:40967)
0808 21:01:01.366767 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.366769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.367046 (+ 277us) server_negotiation.cc:407] Connection header received
0808 21:01:01.367160 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.367162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.367175 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.367205 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.367679 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49028: BlockingRecv error: recv got EOF from 127.0.0.1:49028 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 21:01:01.402325 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.401457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.15.232.189:40967)
0808 21:01:01.401518 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.401520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.401670 (+ 150us) server_negotiation.cc:407] Connection header received
0808 21:01:01.401732 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.401733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.401745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.401819 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.402238 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49040: BlockingRecv error: recv got EOF from 127.0.0.1:49040 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260808 21:01:01.468703 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.467841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49056 (local address 127.15.232.189:40967)
0808 21:01:01.467967 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.467969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.467988 (+ 19us) server_negotiation.cc:407] Connection header received
0808 21:01:01.468067 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.468068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.468080 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.468148 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.468613 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49056: BlockingRecv error: recv got EOF from 127.0.0.1:49056 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260808 21:01:01.599115 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.598249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49072 (local address 127.15.232.189:40967)
0808 21:01:01.598349 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.598351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.598415 (+ 64us) server_negotiation.cc:407] Connection header received
0808 21:01:01.598481 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.598482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.598511 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.598586 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.599018 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49072: BlockingRecv error: recv got EOF from 127.0.0.1:49072 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 21:01:01.857314 16276 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:01.856564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.15.232.189:40967)
0808 21:01:01.856649 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:01.856651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:01.856663 (+ 12us) server_negotiation.cc:407] Connection header received
0808 21:01:01.856744 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:01.856745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:01.856769 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:01.856836 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:01.857227 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49078: BlockingRecv error: recv got EOF from 127.0.0.1:49078 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 21:01:02.190856 16429 catalog_manager.cc:1259] Loaded cluster ID: bc97ad6daaf94f7292e7750e716427a0
I20260808 21:01:02.190996 16429 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6: loading cluster ID for follower catalog manager: success
I20260808 21:01:02.191919 16429 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6: acquiring CA information for follower catalog manager: success
I20260808 21:01:02.192251 16429 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 02568a128aee41ef8445e3e67abd8ae6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 21:01:02.206264 16450 catalog_manager.cc:1259] Loaded cluster ID: bc97ad6daaf94f7292e7750e716427a0
I20260808 21:01:02.206350 16450 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c: loading cluster ID for follower catalog manager: success
I20260808 21:01:02.207238 16450 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c: acquiring CA information for follower catalog manager: success
I20260808 21:01:02.207602 16450 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 42749c7108184706be1fff1504531b6c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 21:01:02.373471 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:02.370709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.15.232.189:40967)
0808 21:01:02.370854 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:02.370857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:02.370871 (+ 14us) server_negotiation.cc:407] Connection header received
0808 21:01:02.370905 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:02.370906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:02.370983 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:02.371084 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:02.371244 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.371794 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.372596 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.372808 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.372890 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:02.373083 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:02.373205 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:02.373297 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:02.373340 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 21:01:02.373363 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260808 21:01:02.377781 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:02.375497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.15.232.188:36189)
0808 21:01:02.375592 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:02.375594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:02.375622 (+ 28us) server_negotiation.cc:407] Connection header received
0808 21:01:02.375659 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:02.375660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:02.375696 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:02.375762 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:02.375869 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.376340 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.377042 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.377241 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.377296 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:02.377423 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:02.377533 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:02.377590 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:02.377638 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 21:01:02.377655 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":43,"threads_started":1}
I20260808 21:01:02.382694 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:02.379607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.15.232.188:36189)
0808 21:01:02.379657 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:02.379659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:02.379889 (+ 230us) server_negotiation.cc:407] Connection header received
0808 21:01:02.380000 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:02.380001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:02.380011 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:02.380063 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:02.380267 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.380746 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.381840 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.382134 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.382192 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:02.382402 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:02.382431 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:02.382496 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:02.382591 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 21:01:02.382617 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 21:01:02.382730 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:02.379567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.15.232.189:40967)
0808 21:01:02.379668 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:02.379671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:02.379924 (+ 253us) server_negotiation.cc:407] Connection header received
0808 21:01:02.379985 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:02.379986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:02.379999 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:02.380050 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:02.380191 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.380934 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.381950 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.382243 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.382313 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:02.382447 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:02.382493 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:02.382547 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:02.382630 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 21:01:02.382659 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 21:01:02.382800 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:02.379550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.15.232.190:34185)
0808 21:01:02.379668 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:02.379671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:02.379792 (+ 121us) server_negotiation.cc:407] Connection header received
0808 21:01:02.379878 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:02.379879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:02.379933 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:02.380023 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:02.380223 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.380978 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.381738 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:02.382038 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:02.382105 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:02.382318 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:02.382434 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:02.382495 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:02.382683 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 21:01:02.382709 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260808 21:01:02.384570 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16059
I20260808 21:01:02.395799 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16171
I20260808 21:01:02.412292 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16285
2026-08-08T21:01:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3234 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 21:01:02 dist-test-slave-c12x krb5kdc[16637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 21:01:02 dist-test-slave-c12x krb5kdc[16637](info): set up 2 sockets
Aug 08 21:01:02 dist-test-slave-c12x krb5kdc[16637](info): commencing operation
krb5kdc: starting...
W20260808 21:01:03.762815 16290 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/test-user.keytab.
Aug 08 21:01:03 dist-test-slave-c12x krb5kdc[16637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T21:01:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T21:01:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.232.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
Entry for principal kudu/127.15.232.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab.
I20260808 21:01:03.908653 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37755
--webserver_interface=127.15.232.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:37755,127.15.232.189:34113,127.15.232.188:46231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 21:01:04.001529 16660 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 21:01:04.001802 16660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:01:04.001824 16660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:01:04.001852 16660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:01:04.003070 16660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:01:04.003127 16660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:01:04.003145 16660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:01:04.003161 16660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:01:04.003175 16660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:01:04.005034 16660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:37755,127.15.232.189:34113,127.15.232.188:46231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.190.keytab
--principal=kudu/127.15.232.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.190:37755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.232.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:01:04.005653 16660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:01:04.005892 16660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:01:04.011564 16671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:04.011682 16666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:04.011607 16668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:01:04.011924 16660 server_base.cc:1034] running on GCE node
I20260808 21:01:04.012198 16660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:01:04.012564 16660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:01:04.013764 16660 hybrid_clock.cc:648] HybridClock initialized: now 1786222864013745 us; error 36 us; skew 500 ppm
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:01:04.020687 16660 init.cc:377] Logged in from keytab as kudu/127.15.232.190@KRBTEST.COM (short username kudu)
I20260808 21:01:04.021229 16660 webserver.cc:460] Webserver started at http://127.15.232.190:33585/ using document root <none> and password file <none>
I20260808 21:01:04.021490 16660 fs_manager.cc:359] Metadata directory not provided
I20260808 21:01:04.021564 16660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:01:04.021723 16660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:01:04.022917 16660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/data/instance:
uuid: "3ab4d2f4100d4a87b0a561c1fef7598a"
format_stamp: "Formatted at 2026-08-08 21:01:04 on dist-test-slave-c12x"
server_key: "6ebac9ae80768d6bcceee464a1501921"
server_key_iv: "3c7cd81612ff860f7ec662cb1011d7b4"
server_key_version: "encryptionkey@0"
I20260808 21:01:04.023303 16660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/wal/instance:
uuid: "3ab4d2f4100d4a87b0a561c1fef7598a"
format_stamp: "Formatted at 2026-08-08 21:01:04 on dist-test-slave-c12x"
server_key: "6ebac9ae80768d6bcceee464a1501921"
server_key_iv: "3c7cd81612ff860f7ec662cb1011d7b4"
server_key_version: "encryptionkey@0"
I20260808 21:01:04.024950 16660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 21:01:04.026047 16681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:01:04.026347 16660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:01:04.026549 16660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/wal
uuid: "3ab4d2f4100d4a87b0a561c1fef7598a"
format_stamp: "Formatted at 2026-08-08 21:01:04 on dist-test-slave-c12x"
server_key: "6ebac9ae80768d6bcceee464a1501921"
server_key_iv: "3c7cd81612ff860f7ec662cb1011d7b4"
server_key_version: "encryptionkey@0"
I20260808 21:01:04.026652 16660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:01:04.045145 16660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:01:04.048450 16660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:01:04.048684 16660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:01:04.053583 16660 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.190:37755
I20260808 21:01:04.053622 16757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.190:37755 every 8 connection(s)
I20260808 21:01:04.054034 16660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/data/info.pb
I20260808 21:01:04.055017 16758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:01:04.056454 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16660
I20260808 21:01:04.056576 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-0/wal/instance
I20260808 21:01:04.056787 16290 external_mini_cluster.cc:1428] Setting key 4490e384aa5ca741e6c4ce4e8b7a330b
I20260808 21:01:04.056767 16758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.189@KRBTEST.COM; defaulting to no policy
I20260808 21:01:04.069886 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.057612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51866 (local address 127.15.232.190:37755)
0808 21:01:04.057806 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.057814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.057901 (+ 87us) server_negotiation.cc:407] Connection header received
0808 21:01:04.058917 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.058920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.058948 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.059087 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.059781 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.060959 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.062576 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.062925 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.063023 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.065959 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.065968 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.065968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.065975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.067919 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.068497 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.068500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.068501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.068592 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.068833 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.068834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.068834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.069091 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.069176 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.069604 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.069805 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260808 21:01:04.069905 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.057537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37755 (local address 127.0.0.1:51866)
0808 21:01:04.057756 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.057810 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:04.058666 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:04.059125 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:04.059129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:04.059407 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:04.059657 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.059663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.061073 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.061074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.062469 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.062472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.063034 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.063036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.063146 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.063943 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:04.063955 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:04.065812 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:04.068155 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.068163 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.068165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.068388 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.068659 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.068659 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.068660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.068767 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.069294 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:04.069320 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:04.069409 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 21:01:04.069525 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 21:01:04.069683 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.189@KRBTEST.COM" created.
I20260808 21:01:04.070816 16758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } has no permanent_uuid. Determining permanent_uuid...
I20260808 21:01:04.071288 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.071156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:34113 (local address 127.0.0.1:57848)
0808 21:01:04.071210 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.071265 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 21:01:04.071519 16688 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
I20260808 21:01:04.072045 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.071921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:34113 (local address 127.0.0.1:57864)
0808 21:01:04.071991 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.072024 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 21:01:04.072196 16758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.189:34113: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab.
Entry for principal kudu/127.15.232.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab.
I20260808 21:01:04.087193 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.189:34113
--webserver_interface=127.15.232.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab
--principal=kudu/127.15.232.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:37755,127.15.232.189:34113,127.15.232.188:46231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 21:01:04.121459 16758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } attempt: 1
I20260808 21:01:04.122145 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.121977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:34113 (local address 127.0.0.1:57874)
0808 21:01:04.122067 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.122113 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 21:01:04.122591 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.122495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:34113 (local address 127.0.0.1:57886)
0808 21:01:04.122540 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.122570 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 21:01:04.122709 16758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.189:34113: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
W20260808 21:01:04.176775 16769 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 21:01:04.177064 16769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:01:04.177086 16769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:01:04.177112 16769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:01:04.178313 16769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:01:04.178371 16769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:01:04.178388 16769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:01:04.178403 16769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:01:04.178418 16769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:01:04.180099 16769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:37755,127.15.232.189:34113,127.15.232.188:46231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.189.keytab
--principal=kudu/127.15.232.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.189:34113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.232.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:01:04.180336 16769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:01:04.180575 16769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:01:04.186553 16779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:01:04.186587 16769 server_base.cc:1034] running on GCE node
W20260808 21:01:04.186617 16786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:04.186553 16780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:01:04.187237 16769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:01:04.187654 16769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:01:04.188855 16769 hybrid_clock.cc:648] HybridClock initialized: now 1786222864188842 us; error 33 us; skew 500 ppm
I20260808 21:01:04.190856 16758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } attempt: 2
I20260808 21:01:04.191363 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.191220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:34113 (local address 127.0.0.1:57894)
0808 21:01:04.191285 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.191326 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:01:04.191696 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.191628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:34113 (local address 127.0.0.1:57906)
0808 21:01:04.191655 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.191680 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260808 21:01:04.191811 16758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.189:34113: Network error: Client connection negotiation failed: client connection to 127.15.232.189:34113: connect: Connection refused (error 111)
I20260808 21:01:04.192001 16769 init.cc:377] Logged in from keytab as kudu/127.15.232.189@KRBTEST.COM (short username kudu)
I20260808 21:01:04.192570 16769 webserver.cc:460] Webserver started at http://127.15.232.189:35053/ using document root <none> and password file <none>
I20260808 21:01:04.192852 16769 fs_manager.cc:359] Metadata directory not provided
I20260808 21:01:04.192924 16769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:01:04.193075 16769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:01:04.194213 16769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/data/instance:
uuid: "83e027f685e942978d25c46a8c997370"
format_stamp: "Formatted at 2026-08-08 21:01:04 on dist-test-slave-c12x"
server_key: "f9d0dd9d02fa584824cc7f779e0a32a0"
server_key_iv: "dd352fc975042133ed701c727c5c7a45"
server_key_version: "encryptionkey@0"
I20260808 21:01:04.194615 16769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/wal/instance:
uuid: "83e027f685e942978d25c46a8c997370"
format_stamp: "Formatted at 2026-08-08 21:01:04 on dist-test-slave-c12x"
server_key: "f9d0dd9d02fa584824cc7f779e0a32a0"
server_key_iv: "dd352fc975042133ed701c727c5c7a45"
server_key_version: "encryptionkey@0"
I20260808 21:01:04.196558 16769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 21:01:04.197523 16802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:01:04.197737 16769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:01:04.197842 16769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/wal
uuid: "83e027f685e942978d25c46a8c997370"
format_stamp: "Formatted at 2026-08-08 21:01:04 on dist-test-slave-c12x"
server_key: "f9d0dd9d02fa584824cc7f779e0a32a0"
server_key_iv: "dd352fc975042133ed701c727c5c7a45"
server_key_version: "encryptionkey@0"
I20260808 21:01:04.197933 16769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:01:04.215229 16769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:01:04.218231 16769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:01:04.218437 16769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:01:04.223193 16769 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.189:34113
I20260808 21:01:04.223245 16876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.189:34113 every 8 connection(s)
I20260808 21:01:04.223750 16769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/data/info.pb
I20260808 21:01:04.225104 16878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:01:04.226852 16878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:01:04.233727 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16769
I20260808 21:01:04.233843 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-1/wal/instance
I20260808 21:01:04.234050 16290 external_mini_cluster.cc:1428] Setting key d3faf7b728d072620ee6555db420188a
I20260808 21:01:04.237799 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.227670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51878 (local address 127.15.232.190:37755)
0808 21:01:04.227777 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.227779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.228193 (+ 414us) server_negotiation.cc:407] Connection header received
0808 21:01:04.228741 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.228742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.228771 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.228832 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.229481 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.230047 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.231206 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.231582 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.231658 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.234242 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.234243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.234243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.234252 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.236052 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.236655 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.236656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.236656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.236716 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.237050 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.237051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.237051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.237197 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.237267 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.237638 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.237731 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 21:01:04.238068 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.227647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37755 (local address 127.0.0.1:51878)
0808 21:01:04.228048 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.228087 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:04.228591 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:04.228929 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:04.228932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:04.229205 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:04.229404 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.229409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.230161 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.230162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.231104 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.231107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.231710 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.231711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.231828 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.232450 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:04.232459 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:04.234130 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:04.236160 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.236170 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.236173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.236566 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.236791 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.236791 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.236792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.236947 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.237279 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:04.237285 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:04.237482 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 21:01:04.237602 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 21:01:04.237789 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":141,"threads_started":1}
I20260808 21:01:04.238660 16878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.232.188@KRBTEST.COM; defaulting to no policy
I20260808 21:01:04.246316 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.238972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:34113 (local address 127.0.0.1:57910)
0808 21:01:04.239084 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.239093 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:04.239125 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:04.239347 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:04.239348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:04.239529 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:04.239587 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.239588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.240462 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.240463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.240961 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.240962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.241438 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.241439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.241504 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.242022 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:04.242026 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:04.243527 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:04.245170 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.245171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.245172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.245333 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.245634 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.245635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.245635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.245696 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.246040 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:04.246041 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:04.246084 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 21:01:04.246154 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 21:01:04.246245 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 21:01:04.246481 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.239025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57910 (local address 127.15.232.189:34113)
0808 21:01:04.239219 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.239224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.239240 (+ 16us) server_negotiation.cc:407] Connection header received
0808 21:01:04.239256 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.239258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.239268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.239347 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.239671 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.240361 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.241045 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.241363 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.241442 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.243615 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.243624 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.243625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.243633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.245090 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.245480 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.245483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.245483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.245550 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.245759 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.245760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.245760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.245937 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.246021 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.246202 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.246320 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260808 21:01:04.247380 16878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.232.188@KRBTEST.COM" created.
I20260808 21:01:04.248024 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.247825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36362)
0808 21:01:04.247938 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.247999 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 21:01:04.248201 16804 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
I20260808 21:01:04.248745 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.248611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36366)
0808 21:01:04.248674 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.248712 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 21:01:04.248935 16878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:46231: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.232.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab.
Entry for principal kudu/127.15.232.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab.
I20260808 21:01:04.267259 16290 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0tXHI2/build/release/bin/kudu
/tmp/dist-test-task0tXHI2/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.188:46231
--webserver_interface=127.15.232.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab
--principal=kudu/127.15.232.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.232.148:44119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.232.190:37755,127.15.232.189:34113,127.15.232.188:46231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 21:01:04.282994 16758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } attempt: 3
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
I20260808 21:01:04.291249 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.283420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:34113 (local address 127.0.0.1:57926)
0808 21:01:04.283535 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.283547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:04.283624 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:04.283910 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:04.283911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:04.284117 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:04.284186 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.284188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.285019 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.285021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.285585 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.285587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.285983 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.285984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.286066 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.286602 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:04.286607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:04.288142 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:04.290063 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.290064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.290065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.290266 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.290506 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.290507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.290507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.290577 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.290868 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:04.290869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:04.290922 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 21:01:04.291008 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 21:01:04.291129 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 21:01:04.291249 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.283539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.15.232.189:34113)
0808 21:01:04.283610 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.283612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.283625 (+ 13us) server_negotiation.cc:407] Connection header received
0808 21:01:04.283770 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.283773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.283795 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.283900 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.284287 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.284908 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.285675 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.285908 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.285985 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.288302 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.288304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.288304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.288325 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.289897 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.290352 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.290353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.290354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.290421 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.290651 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.290653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.290653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.290784 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.290863 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.291049 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.291132 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 21:01:04.291939 16758 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } has no permanent_uuid. Determining permanent_uuid...
I20260808 21:01:04.292351 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.292225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36370)
0808 21:01:04.292283 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.292315 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 21:01:04.292771 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.292668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36374)
0808 21:01:04.292720 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.292748 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 21:01:04.292902 16758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:46231: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
I20260808 21:01:04.298203 16878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } attempt: 1
I20260808 21:01:04.298719 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.298539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36382)
0808 21:01:04.298618 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.298680 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 21:01:04.299180 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.299061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36390)
0808 21:01:04.299125 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.299157 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 21:01:04.299284 16878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:46231: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
I20260808 21:01:04.324060 16758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } attempt: 1
I20260808 21:01:04.324551 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.324411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36406)
0808 21:01:04.324482 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.324521 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 21:01:04.324920 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.324825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36414)
0808 21:01:04.324869 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.324896 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 21:01:04.325018 16758 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:46231: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
W20260808 21:01:04.355767 16888 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 21:01:04.355994 16888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 21:01:04.356014 16888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 21:01:04.356097 16888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 21:01:04.357301 16888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 21:01:04.357342 16888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 21:01:04.357357 16888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 21:01:04.357371 16888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 21:01:04.357432 16888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 21:01:04.358942 16888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.232.148:44119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.232.190:37755,127.15.232.189:34113,127.15.232.188:46231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/krb5kdc/kudu_127.15.232.188.keytab
--principal=kudu/127.15.232.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.232.188:46231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.232.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 20:09:30 UTC on ccycloud.quasar-nuvkld.root.comops.site
build id 4575432
I20260808 21:01:04.359229 16888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 21:01:04.359503 16888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 21:01:04.364820 16896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:04.364826 16895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 21:01:04.364849 16903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 21:01:04.365073 16888 server_base.cc:1034] running on GCE node
I20260808 21:01:04.365331 16888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 21:01:04.365660 16888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 21:01:04.366815 16888 hybrid_clock.cc:648] HybridClock initialized: now 1786222864366790 us; error 44 us; skew 500 ppm
I20260808 21:01:04.367455 16878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } attempt: 2
I20260808 21:01:04.367940 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.367768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36424)
0808 21:01:04.367858 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.367901 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 21:01:04.368335 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.368225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36430)
0808 21:01:04.368275 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.368307 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 21:01:04.368444 16878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.232.188:46231: Network error: Client connection negotiation failed: client connection to 127.15.232.188:46231: connect: Connection refused (error 111)
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 21:01:04.369922 16888 init.cc:377] Logged in from keytab as kudu/127.15.232.188@KRBTEST.COM (short username kudu)
I20260808 21:01:04.370386 16888 webserver.cc:460] Webserver started at http://127.15.232.188:46645/ using document root <none> and password file <none>
I20260808 21:01:04.370654 16888 fs_manager.cc:359] Metadata directory not provided
I20260808 21:01:04.370728 16888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 21:01:04.370882 16888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 21:01:04.372179 16888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/data/instance:
uuid: "95535ed25ff94aef92630da64fdbe62d"
format_stamp: "Formatted at 2026-08-08 21:01:04 on dist-test-slave-c12x"
server_key: "bc2aefff6d305a37db60efb5182628c8"
server_key_iv: "e66a249223ac085112486065a0de1cdc"
server_key_version: "encryptionkey@0"
I20260808 21:01:04.372593 16888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/wal/instance:
uuid: "95535ed25ff94aef92630da64fdbe62d"
format_stamp: "Formatted at 2026-08-08 21:01:04 on dist-test-slave-c12x"
server_key: "bc2aefff6d305a37db60efb5182628c8"
server_key_iv: "e66a249223ac085112486065a0de1cdc"
server_key_version: "encryptionkey@0"
I20260808 21:01:04.374003 16888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 21:01:04.374899 16915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 21:01:04.375123 16888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 21:01:04.375255 16888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/data,/tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/wal
uuid: "95535ed25ff94aef92630da64fdbe62d"
format_stamp: "Formatted at 2026-08-08 21:01:04 on dist-test-slave-c12x"
server_key: "bc2aefff6d305a37db60efb5182628c8"
server_key_iv: "e66a249223ac085112486065a0de1cdc"
server_key_version: "encryptionkey@0"
I20260808 21:01:04.375344 16888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 21:01:04.384480 16888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 21:01:04.387063 16888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 21:01:04.387253 16888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 21:01:04.391676 16888 rpc_server.cc:307] RPC server started. Bound to: 127.15.232.188:46231
I20260808 21:01:04.391728 16997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.232.188:46231 every 8 connection(s)
I20260808 21:01:04.392122 16888 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/data/info.pb
I20260808 21:01:04.393076 16998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 21:01:04.393642 16290 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0tXHI2/build/release/bin/kudu as pid 16888
I20260808 21:01:04.393770 16290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0tXHI2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786222573557010-16290-0/minicluster-data/master-2/wal/instance
I20260808 21:01:04.393980 16290 external_mini_cluster.cc:1428] Setting key 9600c5d5471a701df14ac59f320c02e2
I20260808 21:01:04.395258 16998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } has no permanent_uuid. Determining permanent_uuid...
I20260808 21:01:04.400163 16758 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } attempt: 2
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.190@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
I20260808 21:01:04.405854 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.396300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.15.232.190:37755)
0808 21:01:04.396354 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.396356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.396508 (+ 152us) server_negotiation.cc:407] Connection header received
0808 21:01:04.396540 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.396541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.396552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.396625 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.397083 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.397795 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.399140 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.399420 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.399540 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.402433 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.402434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.402435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.402443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.404217 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.404827 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.404829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.404830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.404902 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.405179 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.405180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.405181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.405300 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.405387 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.405673 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.405762 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 21:01:04.406076 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.396156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.190:37755 (local address 127.0.0.1:51886)
0808 21:01:04.396359 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.396413 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:04.396483 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:04.396636 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:04.396638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:04.396899 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:04.396993 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.396997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.397956 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.397958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.399052 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.399055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.399581 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.399582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.399660 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.400346 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:04.400358 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:04.402341 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:04.404302 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.404315 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.404318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.404728 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.404972 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.404973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.404973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.405100 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.405370 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:04.405378 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:04.405532 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 21:01:04.405639 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 21:01:04.405794 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":126,"threads_started":1}
I20260808 21:01:04.406651 16998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } has no permanent_uuid. Determining permanent_uuid...
I20260808 21:01:04.407318 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.394668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36444 (local address 127.15.232.188:46231)
0808 21:01:04.394883 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.394892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.394921 (+ 29us) server_negotiation.cc:407] Connection header received
0808 21:01:04.395387 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.395394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.395519 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.395653 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.396543 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.397857 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.398983 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.399432 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.399567 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.403155 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.403164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.403164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.403172 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.405366 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.405863 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.405866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.405867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.406173 (+ 306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.406396 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.406397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.406398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.406591 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.406777 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.407126 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.407212 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260808 21:01:04.410167 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.400479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36448)
0808 21:01:04.400524 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.400538 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:04.400580 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:04.400890 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:04.400891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:04.401078 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:04.401128 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.401130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.402933 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.402934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.403805 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.403808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.404216 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.404217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.404285 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.404648 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:04.404654 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:04.406152 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:04.409085 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.409086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.409087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.409303 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.409522 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.409523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.409523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.409585 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.409881 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:04.409882 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:04.409925 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 21:01:04.409978 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 21:01:04.410081 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 21:01:04.410607 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.400541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36448 (local address 127.15.232.188:46231)
0808 21:01:04.400750 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.400752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.400759 (+ 7us) server_negotiation.cc:407] Connection header received
0808 21:01:04.400784 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.400785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.400793 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.400868 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.402054 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.402820 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.403894 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.404135 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.404205 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.406418 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.406419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.406420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.406426 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.408995 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.409379 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.409380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.409381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.409464 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.409663 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.409664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.409664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.409777 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.410310 (+ 533us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.410354 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.410466 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":133,"threads_started":1}
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
I20260808 21:01:04.412343 16758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a: Bootstrap starting.
I20260808 21:01:04.413470 16758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a: Neither blocks nor log segments found. Creating new log.
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.190@KRBTEST.COM
I20260808 21:01:04.413836 16758 log.cc:824] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a: Log is configured to *not* fsync() on all Append() calls
I20260808 21:01:04.414826 16758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a: No bootstrap required, opened a new log
I20260808 21:01:04.415122 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.407079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.15.232.189:34113)
0808 21:01:04.407130 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.407132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.407175 (+ 43us) server_negotiation.cc:407] Connection header received
0808 21:01:04.407282 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.407283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.407294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.407324 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.407692 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.408336 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.409437 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.409761 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.409834 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.411923 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.411924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.411925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.411934 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.413619 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.414040 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.414042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.414042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.414106 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.414328 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.414328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.414329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.414487 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.414556 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.414755 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.414837 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 21:01:04.415668 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.406948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.189:34113 (local address 127.0.0.1:57928)
0808 21:01:04.407010 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.407023 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:04.407199 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:04.407395 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:04.407395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:04.407584 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:04.407624 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.407626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.408420 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.408421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.409368 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.409371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.409839 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.409840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.409910 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.410355 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:04.410360 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:04.411823 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:04.413727 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.413728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.413729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.413944 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.414187 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.414188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.414188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.414246 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.414587 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:04.414588 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:04.414635 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 21:01:04.415426 (+ 791us) client_negotiation.cc:240] Negotiation successful
0808 21:01:04.415569 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 21:01:04.416078 16998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } has no permanent_uuid. Determining permanent_uuid...
I20260808 21:01:04.417316 16758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } }
I20260808 21:01:04.417491 16758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab4d2f4100d4a87b0a561c1fef7598a, State: Initialized, Role: FOLLOWER
I20260808 21:01:04.417589 16758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } }
I20260808 21:01:04.417737 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.408501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.15.232.190:37755)
0808 21:01:04.408542 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.408544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.408574 (+ 30us) server_negotiation.cc:407] Connection header received
0808 21:01:04.408662 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.408662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.408671 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.408698 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.409318 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.409990 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.411173 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.411374 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.411405 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.414515 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.414516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.414516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.414525 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.416311 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.416760 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.416761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.416762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.416827 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.417084 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.417084 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.417085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.417218 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.417371 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.417484 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.417581 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 21:01:04.417994 17007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.418110 17007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:04.418000 16758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [sys.catalog]: configured and running, proceeding with master startup.
W20260808 21:01:04.420599 17027 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:01:04.420821 17027 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.188@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
I20260808 21:01:04.421833 16660 master_runner.cc:186] Error getting master registration for 127.15.232.189:34113: OK, instance_id { permanent_uuid: "83e027f685e942978d25c46a8c997370" instance_seqno: 1786222864221853 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 21:01:04.425105 16660 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.232.189@KRBTEST.COM
I20260808 21:01:04.426143 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.416354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36462)
0808 21:01:04.416512 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.416523 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:04.416561 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:04.418171 (+ 1610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:04.418172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:04.418316 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:04.418356 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.418358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.419260 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.419261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.420049 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.420052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.420536 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.420537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.420586 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.421078 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:04.421083 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:04.422590 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:04.424546 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.424547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.424547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.425027 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.425408 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.425409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.425410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.425484 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.425862 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:04.425863 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:04.425921 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 21:01:04.425972 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 21:01:04.426074 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260808 21:01:04.426496 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.416416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36462 (local address 127.15.232.188:46231)
0808 21:01:04.418037 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.418039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.418049 (+ 10us) server_negotiation.cc:407] Connection header received
0808 21:01:04.418072 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.418073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.418085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.418431 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.418455 (+ 24us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.419177 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.420137 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.420453 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.420744 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.422684 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.422686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.422686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.422694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.424450 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.425255 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.425256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.425256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.425322 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.425568 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.425569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.425569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.425779 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.426272 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.426315 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.426395 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1598}
I20260808 21:01:04.427971 16998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d: Bootstrap starting.
I20260808 21:01:04.428834 16998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d: Neither blocks nor log segments found. Creating new log.
I20260808 21:01:04.429183 16998 log.cc:824] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d: Log is configured to *not* fsync() on all Append() calls
I20260808 21:01:04.430034 16998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d: No bootstrap required, opened a new log
I20260808 21:01:04.430135 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.418474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57930 (local address 127.15.232.189:34113)
0808 21:01:04.418575 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.418577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.418584 (+ 7us) server_negotiation.cc:407] Connection header received
0808 21:01:04.418603 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.418604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.418612 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.418671 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.419423 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.420166 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.421118 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.421419 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.421481 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.426033 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.426035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.426036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.426044 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.428267 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.428702 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.428703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.428703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.428775 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.429029 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.429031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.429032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.429210 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.429419 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.429934 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.430027 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 21:01:04.432118 16290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 21:01:04.432267 16998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } }
I20260808 21:01:04.432420 16998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95535ed25ff94aef92630da64fdbe62d, State: Initialized, Role: FOLLOWER
I20260808 21:01:04.432492 16998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } }
I20260808 21:01:04.432821 16998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:01:04.432796 17032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.433125 17032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:04.433233 16724 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51900
I20260808 21:01:04.433403 16724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 21:01:04.433465 16724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:01:04.434706 16724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } }
I20260808 21:01:04.434995 16724 leader_election.cc:288] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [CANDIDATE]: Term 1 election: Requested vote from peers 83e027f685e942978d25c46a8c997370 (127.15.232.189:34113), 95535ed25ff94aef92630da64fdbe62d (127.15.232.188:46231)
W20260808 21:01:04.435204 17046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:01:04.435307 17046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 21:01:04.435567 16960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "95535ed25ff94aef92630da64fdbe62d"
I20260808 21:01:04.435694 16960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:01:04.436707 16960 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab4d2f4100d4a87b0a561c1fef7598a in term 1.
I20260808 21:01:04.436932 16687 leader_election.cc:302] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab4d2f4100d4a87b0a561c1fef7598a, 95535ed25ff94aef92630da64fdbe62d; no voters:
I20260808 21:01:04.437079 17007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 21:01:04.437266 17007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [term 1 LEADER]: Becoming Leader. State: Replica: 3ab4d2f4100d4a87b0a561c1fef7598a, State: Running, Role: LEADER
I20260808 21:01:04.437371 17007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } }
I20260808 21:01:04.437855 17007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ab4d2f4100d4a87b0a561c1fef7598a. Latest consensus state: current_term: 1 leader_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.437985 17007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [sys.catalog]: This master's current role is: LEADER
I20260808 21:01:04.436615 16888 master_runner.cc:186] Error getting master registration for 127.15.232.189:34113: OK, instance_id { permanent_uuid: "83e027f685e942978d25c46a8c997370" instance_seqno: 1786222864221853 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 21:01:04.439028 16888 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 21:01:04.445075 17057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 21:01:04.445338 17057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 21:01:04.447278 16960 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [term 1 FOLLOWER]: Refusing update from remote peer 3ab4d2f4100d4a87b0a561c1fef7598a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:01:04.447703 17007 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [LEADER]: Connected to new peer: Peer: permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:01:04.449182 17032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ab4d2f4100d4a87b0a561c1fef7598a. Latest consensus state: current_term: 1 leader_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.449290 17032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:04.452036 17032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.452157 17032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:04.451359 17055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.452422 17055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [sys.catalog]: This master's current role is: LEADER
I20260808 21:01:04.452521 17063 mvcc.cc:205] Tried to move back new op lower bound from 7316368852772999168 to 7316368852736294912. Current Snapshot: MvccSnapshot[applied={T|T < 7316368852772999168}]
I20260808 21:01:04.452718 17064 mvcc.cc:205] Tried to move back new op lower bound from 7316368852772999168 to 7316368852736294912. Current Snapshot: MvccSnapshot[applied={T|T < 7316368852772999168}]
I20260808 21:01:04.453658 17057 catalog_manager.cc:1347] Generated new cluster ID: 6a2c73f4146745888416809b70d84851
I20260808 21:01:04.453719 17057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 21:01:04.457234 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.443807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.15.232.190:37755)
0808 21:01:04.443879 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.443882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.444481 (+ 599us) server_negotiation.cc:407] Connection header received
0808 21:01:04.444692 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.444693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.444706 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.444745 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.446350 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.447104 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.448989 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.449333 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.449391 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.453040 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.453042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.453042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.453051 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.455332 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.456092 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.456094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.456094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.456156 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.456492 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.456493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.456493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.456627 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.456711 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.457044 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.457133 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 21:01:04.459561 16878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } attempt: 3
I20260808 21:01:04.462596 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.444034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36464 (local address 127.15.232.188:46231)
0808 21:01:04.444078 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.444081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.451593 (+ 7512us) server_negotiation.cc:407] Connection header received
0808 21:01:04.451700 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.451701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.451712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.451741 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.452401 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.453119 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.454143 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.454504 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.454570 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.457980 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.457982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.457983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.457991 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.460029 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.460431 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.460432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.460432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.460489 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.461871 (+ 1382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.461872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.461873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.462010 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.462109 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.462318 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.462416 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 21:01:04 dist-test-slave-c12x krb5kdc[16637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786222864, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.232.189@KRBTEST.COM for kudu/127.15.232.188@KRBTEST.COM
I20260808 21:01:04.467052 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.444070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57940 (local address 127.15.232.189:34113)
0808 21:01:04.444118 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.444120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.454251 (+ 10131us) server_negotiation.cc:407] Connection header received
0808 21:01:04.454360 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.454361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.454384 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.454415 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.455301 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.456081 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.458297 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.458652 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.458696 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.463337 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.463339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.463339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.463347 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.465244 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.465659 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.465660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.465660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.465714 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.466178 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.466179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.466180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.466321 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.466641 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.466865 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.466949 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 21:01:04.469156 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.459856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.232.188:46231 (local address 127.0.0.1:36470)
0808 21:01:04.459964 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 21:01:04.459975 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 21:01:04.460019 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 21:01:04.460288 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 21:01:04.460288 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 21:01:04.460483 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 21:01:04.460528 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.460530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.461648 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.461649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.462675 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 21:01:04.462678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.463333 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.463334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 21:01:04.463397 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.463813 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 21:01:04.463818 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 21:01:04.465499 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 21:01:04.467499 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.467500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.467500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.467743 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.468007 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 21:01:04.468008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 21:01:04.468008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 21:01:04.468100 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 21:01:04.468608 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 21:01:04.468609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 21:01:04.468676 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 21:01:04.468759 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 21:01:04.468940 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 21:01:04.469658 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:04.460102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.15.232.188:46231)
0808 21:01:04.460147 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.460149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.460156 (+ 7us) server_negotiation.cc:407] Connection header received
0808 21:01:04.460178 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.460179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.460189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.460241 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.460623 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.461537 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.462833 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:04.463207 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:04.463275 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:04.465609 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 21:01:04.465611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 21:01:04.465611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 21:01:04.465618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 21:01:04.467372 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.467835 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.467836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.467837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.467912 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 21:01:04.468220 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 21:01:04.468222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 21:01:04.468222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 21:01:04.468479 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 21:01:04.469082 (+ 603us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:04.469237 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 21:01:04.469500 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 21:01:04.471500 16878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370: Bootstrap starting.
I20260808 21:01:04.472364 16878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370: Neither blocks nor log segments found. Creating new log.
I20260808 21:01:04.472726 16878 log.cc:824] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370: Log is configured to *not* fsync() on all Append() calls
I20260808 21:01:04.473892 16878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370: No bootstrap required, opened a new log
I20260808 21:01:04.476794 16878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } }
I20260808 21:01:04.476979 16878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83e027f685e942978d25c46a8c997370, State: Initialized, Role: FOLLOWER
I20260808 21:01:04.477080 16878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } }
I20260808 21:01:04.477672 16878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 21:01:04.477734 16843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "83e027f685e942978d25c46a8c997370"
I20260808 21:01:04.477860 16843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [term 0 FOLLOWER]: Advancing to term 1
I20260808 21:01:04.478847 16843 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ab4d2f4100d4a87b0a561c1fef7598a in term 1.
I20260808 21:01:04.479202 16842 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [term 1 FOLLOWER]: Refusing update from remote peer 3ab4d2f4100d4a87b0a561c1fef7598a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 21:01:04.479533 17055 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [LEADER]: Connected to new peer: Peer: permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 21:01:04.487600 17090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.487955 17090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:04.488072 17090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ab4d2f4100d4a87b0a561c1fef7598a. Latest consensus state: current_term: 1 leader_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.488686 17090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:04.488667 17091 mvcc.cc:205] Tried to move back new op lower bound from 7316368852772999168 to 7316368852736294912. Current Snapshot: MvccSnapshot[applied={T|T < 7316368852772999168}]
W20260808 21:01:04.488670 17088 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 21:01:04.488955 17088 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 21:01:04.488884 17090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.489063 17090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 21:01:04.501952 17055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ab4d2f4100d4a87b0a561c1fef7598a" member_type: VOTER last_known_addr { host: "127.15.232.190" port: 37755 } } peers { permanent_uuid: "83e027f685e942978d25c46a8c997370" member_type: VOTER last_known_addr { host: "127.15.232.189" port: 34113 } } peers { permanent_uuid: "95535ed25ff94aef92630da64fdbe62d" member_type: VOTER last_known_addr { host: "127.15.232.188" port: 46231 } } }
I20260808 21:01:04.502082 17055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a [sys.catalog]: This master's current role is: LEADER
I20260808 21:01:04.506457 17057 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 21:01:04.507318 17057 catalog_manager.cc:1504] Loading token signing keys...
I20260808 21:01:04.513010 17057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ab4d2f4100d4a87b0a561c1fef7598a: Generated new TSK 0
I20260808 21:01:04.513281 17057 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 21:01:04.540540 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.538688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.15.232.189:34113)
0808 21:01:04.538738 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.538741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.539776 (+ 1035us) server_negotiation.cc:407] Connection header received
0808 21:01:04.539907 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.539908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.539921 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.539953 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.540451 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57942: BlockingRecv error: recv got EOF from 127.0.0.1:57942 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 21:01:04.541545 16762 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.539112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51920 (local address 127.15.232.190:37755)
0808 21:01:04.539153 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.539155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.540986 (+ 1831us) server_negotiation.cc:407] Connection header received
0808 21:01:04.541092 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.541093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.541106 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.541137 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.541471 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51920: BlockingRecv error: recv got EOF from 127.0.0.1:51920 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 21:01:04.542460 17001 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.539460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.15.232.188:46231)
0808 21:01:04.539502 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.539504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.541894 (+ 2390us) server_negotiation.cc:407] Connection header received
0808 21:01:04.541997 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.541998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.542010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.542041 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.542401 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36480: BlockingRecv error: recv got EOF from 127.0.0.1:36480 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 21:01:04.545156 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.544320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57952 (local address 127.15.232.189:34113)
0808 21:01:04.544357 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.544359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.544603 (+ 244us) server_negotiation.cc:407] Connection header received
0808 21:01:04.544716 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.544718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.544738 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.544769 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.545106 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57952: BlockingRecv error: recv got EOF from 127.0.0.1:57952 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 21:01:04.548389 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.547552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.15.232.189:34113)
0808 21:01:04.547589 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.547591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.547858 (+ 267us) server_negotiation.cc:407] Connection header received
0808 21:01:04.547960 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.547961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.547972 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.548003 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.548342 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57966: BlockingRecv error: recv got EOF from 127.0.0.1:57966 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 21:01:04.552490 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.551662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57982 (local address 127.15.232.189:34113)
0808 21:01:04.551699 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.551701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.551960 (+ 259us) server_negotiation.cc:407] Connection header received
0808 21:01:04.552064 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.552065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.552076 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.552107 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.552443 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57982: BlockingRecv error: recv got EOF from 127.0.0.1:57982 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 21:01:04.558907 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.557901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57984 (local address 127.15.232.189:34113)
0808 21:01:04.557947 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.557949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.558253 (+ 304us) server_negotiation.cc:407] Connection header received
0808 21:01:04.558367 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.558368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.558382 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.558416 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.558847 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57984: BlockingRecv error: recv got EOF from 127.0.0.1:57984 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 21:01:04.569485 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.568454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.15.232.189:34113)
0808 21:01:04.568503 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.568505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.568827 (+ 322us) server_negotiation.cc:407] Connection header received
0808 21:01:04.568942 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.568944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.568956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.568989 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.569419 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57998: BlockingRecv error: recv got EOF from 127.0.0.1:57998 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 21:01:04.588150 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.587106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.15.232.189:34113)
0808 21:01:04.587152 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.587155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.587499 (+ 344us) server_negotiation.cc:407] Connection header received
0808 21:01:04.587627 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.587628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.587641 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.587673 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.588087 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58004: BlockingRecv error: recv got EOF from 127.0.0.1:58004 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 21:01:04.622537 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.621722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.15.232.189:34113)
0808 21:01:04.621777 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.621779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.621934 (+ 155us) server_negotiation.cc:407] Connection header received
0808 21:01:04.622008 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.622009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.622023 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.622097 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.622472 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58018: BlockingRecv error: recv got EOF from 127.0.0.1:58018 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 21:01:04.688676 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.687957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.15.232.189:34113)
0808 21:01:04.688015 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.688018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.688111 (+ 93us) server_negotiation.cc:407] Connection header received
0808 21:01:04.688161 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.688162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.688174 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.688255 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.688618 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58020: BlockingRecv error: recv got EOF from 127.0.0.1:58020 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 21:01:04.818985 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:04.818105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58024 (local address 127.15.232.189:34113)
0808 21:01:04.818192 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:04.818195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:04.818296 (+ 101us) server_negotiation.cc:407] Connection header received
0808 21:01:04.818416 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:04.818417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:04.818433 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:04.818486 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:04.818911 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58024: BlockingRecv error: recv got EOF from 127.0.0.1:58024 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 21:01:05.077245 16884 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 21:01:05.076497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58038 (local address 127.15.232.189:34113)
0808 21:01:05.076567 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:05.076570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:05.076581 (+ 11us) server_negotiation.cc:407] Connection header received
0808 21:01:05.076651 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:05.076653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:05.076669 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:05.076746 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 21:01:05.077144 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58038: BlockingRecv error: recv got EOF from 127.0.0.1:58038 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 21:01:05.435823 17046 catalog_manager.cc:1259] Loaded cluster ID: 6a2c73f4146745888416809b70d84851
I20260808 21:01:05.435906 17046 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d: loading cluster ID for follower catalog manager: success
I20260808 21:01:05.437028 17046 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d: acquiring CA information for follower catalog manager: success
I20260808 21:01:05.437355 17046 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 95535ed25ff94aef92630da64fdbe62d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 21:01:05.489549 17088 catalog_manager.cc:1259] Loaded cluster ID: 6a2c73f4146745888416809b70d84851
I20260808 21:01:05.489647 17088 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370: loading cluster ID for follower catalog manager: success
I20260808 21:01:05.490604 17088 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370: acquiring CA information for follower catalog manager: success
I20260808 21:01:05.490897 17088 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 83e027f685e942978d25c46a8c997370: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 21:01:05.593636 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:05.590479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58042 (local address 127.15.232.189:34113)
0808 21:01:05.590636 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:05.590638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:05.590711 (+ 73us) server_negotiation.cc:407] Connection header received
0808 21:01:05.590844 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:05.590845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:05.590919 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:05.591023 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:05.591205 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.591801 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.592674 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.592897 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.592997 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:05.593225 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:05.593352 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:05.593454 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:05.593496 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 21:01:05.593521 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
I20260808 21:01:05.598510 17202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:05.595767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.15.232.188:46231)
0808 21:01:05.595886 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:05.595888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:05.595898 (+ 10us) server_negotiation.cc:407] Connection header received
0808 21:01:05.595962 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:05.595963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:05.596004 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:05.596085 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:05.596258 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.596813 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.597569 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.597782 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.597837 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:05.598020 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:05.598137 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:05.598214 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:05.598289 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 21:01:05.598311 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260808 21:01:05.604090 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:05.600616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58050 (local address 127.15.232.189:34113)
0808 21:01:05.600675 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:05.600678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:05.600878 (+ 200us) server_negotiation.cc:407] Connection header received
0808 21:01:05.600989 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:05.600990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:05.601000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:05.601056 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:05.601203 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.601975 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.603129 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.603558 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.603645 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:05.603791 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:05.603826 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:05.603883 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:05.603951 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 21:01:05.603975 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 21:01:05.604494 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:05.600647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.15.232.190:37755)
0808 21:01:05.600937 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:05.600939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:05.600947 (+ 8us) server_negotiation.cc:407] Connection header received
0808 21:01:05.600994 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:05.600994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:05.601061 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:05.601149 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:05.601300 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.602277 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.603469 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.603758 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.603818 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:05.604108 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:05.604230 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:05.604296 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:05.604380 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 21:01:05.604410 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":83,"threads_started":1}
I20260808 21:01:05.605188 17202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 21:01:05.601029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36484 (local address 127.15.232.188:46231)
0808 21:01:05.601116 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 21:01:05.601118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 21:01:05.601307 (+ 189us) server_negotiation.cc:407] Connection header received
0808 21:01:05.601387 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 21:01:05.601389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 21:01:05.601401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 21:01:05.601467 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 21:01:05.601578 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.602364 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.604215 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 21:01:05.604523 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 21:01:05.604618 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 21:01:05.604929 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 21:01:05.604959 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 21:01:05.605025 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 21:01:05.605093 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 21:01:05.605116 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 21:01:05.606725 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16660
I20260808 21:01:05.621289 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16769
I20260808 21:01:05.635290 16290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0tXHI2/build/release/bin/kudu with pid 16888
2026-08-08T21:01:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3225 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6460 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292078 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST